Method and apparatus for compressing and scaling thumbnails

ABSTRACT

Disclosed are method and apparatus for the manipulation of thumbnail images as used in image-based browsing file management systems. Disclosed are arrangements whereby zooming in and out of thumbnail images can be performed without a continued need to decompress a true image thus providing for faster operation. Pixel interpolation and/or replication are used to generate intermediate images that are display to deliver to the user a perception of a transitory zoom yet are of sufficient detail to maintain user orientation. Aspect ratio zooming of thumbnail containment areas is also disclosed which facilitates ease of browsing. The compression of thumbnail type images using a discrete wavelet transform facilitates the fast zoom of thumbnails and their associated containment areas.

CROSS-REFERENCE TO RELATED APPLICATION

[0001] The present application is a divisional application of U.S.patent application Ser. No. 09/002,883 filed Jan. 5, 1998.

FIELD OF THE INVENTION

[0002] The present invention relates to the display of images on adisplay device and, in particular, to a fast process for zooming-in andout of images, to the layout and associated zooming of image thumbnailsfor image browsing, and to the encoding and scaling of image thumbnailsfor fast display of multiple images at varying resolutions inapplications such as image browsing.

BACKGROUND

[0003] As the price of digital cameras continues to fall, and aspersonal computers become increasingly more powerful, the number ofdigital images managed by home and small business computer users is setto increase dramatically in the very near future. Despite this, the needfor software image browsers to move beyond the limitations of thecurrent design conventions to support effective browsing of manythousands of images has received relatively little attention. Further,despite the large number of digital images commonly encountered byusers, fixed size image thumbnails or image icons continue to be used asthe basis for designing conventional image browsing applications.

[0004] Existing image browsers are based on the concept of displayingfixed size image thumbnails to users. Image thumbnails are small copiesof actual image files that the thumbnails represent. Typically,thumbnails are displayed at between 80×80 pixels and 120×120 pixels.This allows visual display screens to display between 9 and 25thumbnails at any one time depending on the screen and thumbnailresolutions, and the interface design. Consequently, when browsing largenumbers of image thumbnails, users are unable to see more than a smallproportion of the images of interest and are typically forced to scrollthrough vast arrays of thumbnails. This results in significant browsingand orientation problems.

[0005] The fixed resolution at which designers of existing imagebrowsers choose to display thumbnails represents a trade off between thetwo primary uses of thumbnails in image browsers, those being browsingimage content and assessing image content. Browsing image content isdefined here as briefly scanning or glancing at image thumbnails to gainan overview of the images available. Browsing content is limited by thenumber of thumbnails that can be displayed at once. Assessing imagecontent is a closer examination of image thumbnails and the assessmentof detail, and is limited by the resolution of thumbnails that can bedisplayed.

[0006] Conventional approaches to the use of thumbnails have a number ofdisadvantages including the display of only a portion of fixed sizethumbnails in a display area not large enough to display all thethumbnails and the use of scroll bars. Both of these contribute to userdisorientation, general difficulty of navigating an image space, and aninconsistent spatial representation of the image space, amongst otherthings.

[0007] Besides the disadvantages of only displaying only a portion offixed size thumbnails, using scroll bars making navigation of an imagespace difficult, and providing an inconsistent spatial representation ofthe image space, such fixed size thumbnails themselves can use largeamounts of memory or alternatively take relatively long period todecompress from compressed image storage. For example, a conventionalmethod of thumbnail zooming-involves performing progressivedecompressions of the thumbnail image until the desired image isobtained. This provides for a representative depiction of the zoom asseen by the user but involves performing a number of complexdecompressions in succession. This has been found to consume valuableprocessor and operator time which is undesirable.

[0008] Thus, a need clearly exists for a method of providing thumbnailsthat are not fixed in size, have reduced memory requirements, arecapable of rapid scaling, and provide for improved browsingapplications.

SUMMARY

[0009] It is an object of the present invention to substantiallyovercome, or at least ameliorate, one or more problems associated withprior arrangements.

[0010] In accordance with one aspect of the present invention there isdisclosed a method of displaying a representation of a plurality ofimages concurrently on a display device, each image being encoded in ahierarchical compressed format, the hierarchical compressed format beingadapted to provide, without substantially compromising compressionefficiency, a decoding of each encoded image into one or more of aplurality of predetermined sizes, each size definable by a number ofpixels, the method comprising the steps of:

[0011] decoding each encoded image of the plurality of images into atleast one of the plurality of predetermined sizes, wherein a decode timefor each encoded image at each one of the predetermined sizes issubstantially proportional to the number of pixels of each decodedimage; and

[0012] displaying each decoded image of the plurality of imagesconcurrently on the display device.

[0013] In accordance with one aspect of the present invention there isdisclosed a method of displaying a plurality of images concurrently on adisplay device, the method comprising the steps of:

[0014] encoding each image in a hierarchical compressed format capableof providing a plurality of predetermined sizes of each image withoutsubstantially compromising compression efficiency, wherein each image isencoded in a manner so that a decode time for each image at each of thepredetermined sizes is substantially proportional to the number ofpixels in each decoded image, decoding each image of the plurality ofimages, and

[0015] displaying each decoded image of the plurality of imagesconcurrently on said display device.

[0016] Preferably each decoded image is a thumbnail representation ofeach the corresponding encoded image and the encoded image is encodedusing a compression technique employing a linear transform.Advantageously the compression technique substantially encodes groups ofcoefficients representing a resolution of the linear transformsubstantially independently of coefficients representing anotherresolution. Most preferably the linear transform is a discrete wavelettransform and the plurality of predetermined sizes comprises a set ofsizes related by powers of 2. The plurality of predetermined sizes isdesirably the same for each decoded image in said plurality of images.In some implementations the decoding step and the displaying step areinterleaved for each image of the plurality. Also, in someimplementations, a predetermined size of each decoded image is equal toor greater than a desired size of said thumbnail.

[0017] Preferred methods can further comprise the step of downsamplingeach decoded image to provide the desired-size thumbnail if the size ofthe decoded image is greater than the desired size of the thumbnail. Themethods can further comprise the step of upsampling the decoded image toprovide the desired-size thumbnail if the size of the decoded image isless than the desired size of the thumbnail.

[0018] In some implementations the method may comprise the step ofdecoding the image at two different adjacent sizes of a plurality ofsizes to provide said image having a desired size. Desirably the methodfurther comprises the step of interpolating between the two decodedimages having different adjacent sizes to provide the desired-sizethumbnail if the size of one of the two decoded images is greater thanthe desired thumbnail size and the other of the two decoded images isless than the desired thumbnail size.

[0019] Each image is desirably encoded by the steps of:

[0020] transforming said image to derive a plurality of coefficients,each coefficient represented by a predefined bit sequence;

[0021] selecting a portion of said plurality of coefficients as aregion;

[0022] (a) scanning the significance of each bitplane of the selectedregion from a most significant bitplane towards a least significantbitplane and providing a first token in the coded representation foreach insignificant bitplane until a significant bitplane is determined,wherein a second token is provided in the coded representation for thesignificant bitplane;

[0023] (b) partitioning the selected region into two or more subregionshaving a predetermined form, and setting each of the subregions as theselected region;

[0024] (c) repeating steps (a) and (b) commencing from the significantbitplane until the selected region has a predetermined size, wherein thecoefficients of the selected region are coded and provided in the codedrepresentation.

[0025] The transforming step preferably comprises applying a discretewavelet transform to said image, the region comprises said entireplurality of coefficients, and the portion comprises a subband of saidplurality of coefficients. The first and second tokens preferablycomprise bit values of 0 and 1, respectively and the subregions areequally sized, such as square. Desirably the predetermined size of saidsubregion is a 1×1 coefficient and the 1×1 coefficient is encoded byoutputting bits of the corresponding bit sequence beginning with therespective significant bitplane. Preferably, only bits of thecorresponding bit sequence above a predetermined minimum bit level areoutput in the coded representation. Desirably, in step (c), steps (a)and (b) are repeated otherwise until each bitplane of the selectedregion has been scanned. and each bitplane of the selected region abovea minimum bit level has been scanned.

[0026] In other implementations, each image may be encoded by the stepsof:

[0027] decomposing the image using a subband transform to provide anumber of subbands;

[0028] for each subband, selecting the subband as an initial region andperforming the following sub-steps:

[0029] (a) checking if a current bit level of the selected region issignificant;

[0030] (b) if said current bit level is significant, outputting a firsttoken in a coded representation and partitioning the selected regioninto a number of equally sized sub-regions, wherein each sub-region isprocessed as the selected region in turn;

[0031] (c) if said current bit level is insignificant, outputting asecond token in the coded representation and selecting the next lowerbit level of the selected region as said current bit level;

[0032] (d) repeating steps (a) to (c) until the current bit level isless than a specified minimum bit level, or the selected region has apredetermined size and coefficients of the selected region are coded inthe coded representation.

[0033] Desirably, the coefficients of the selected region are coded byrepresenting each coefficient by the bits between the correspondingcurrent bit level and the minimum bit level and the subband transformcomprises a discrete wavelet transform.

[0034] Each image may also be encoded by the steps of:

[0035] a) dividing each image into a plurality of blocks;

[0036] b) applying a subband transform to each of the blocks to providea plurality of AC subband regions, at one or more resolutions, and a DCsubband region for each block;

[0037] c) selecting the DC subband region as a selected region andperforming the following sub-steps:

[0038] ca) checking if a current bitplane of the selected region issignificant;

[0039] cb) if the current bitplane is significant, outputting a firsttoken in a coded representation and partitioning the selected regioninto a number of sub-regions, wherein each sub-region is processed asthe selected region in turn;

[0040] cc) if the current bitplane is insignificant, outputting a secondtoken in the coded representation and selecting the next lower bitplaneof the selected region as the current bitplane;

[0041] cd) repeating sub-steps ca) to cc) until the current bitplane isless than a specified minimum bitplane, or the selected region has apredetermined size and coefficients of the selected region are coded inthe coded representation;

[0042] d) selecting substantially all uncoded AC subband regions as aremaining region of each block and scanning the significance of eachbitplane of the remaining region from a most significant bitplanetowards a least significant bitplane, and outputting the second tokenfor each insignificant bitplane until a significant bitplane isdetermined;

[0043] e) setting one or more AC subband regions, of a currentresolution level, as a selected region and performing the sub-steps ca)to cd);

[0044] f) repeating step e) until substantially all AC subbands of thecurrent resolution level have been encoded; and

[0045] g) repeating steps d) to f) until all AC subbands of each blockhave been encoded.

[0046] The method preferably further comprise the step of caching saidencoded image and/or said decoded image and the step of cachingintermediate size versions of said decoded image.

[0047] Apparatus, system and computer programs for performing thesemethods are also disclosed.

[0048] Many other aspects of the invention and present disclosure willbecome apparent on review and understanding the following descriptionsand claims.

BRIEF DESCRIPTION OF THE DRAWINGS

[0049] One or more embodiments of the present invention will now bedescribed with reference to the accompanying drawings in which:

[0050]FIG. 1 shows an example configuration of a personal computersystem of an embodiment;

[0051]FIG. 2 is a schematic block diagram of the system of FIG. 1;

[0052]FIG. 3 illustrates a thumbnail management system includinghardware and software incorporating the arrangements of FIGS. 1 and 2;

[0053]FIG. 4 illustrates an initial and final stage of zooming up ordown an image in accordance with one embodiment;

[0054]FIG. 5A shows an example of intermediate stages of an image whenzooming up;

[0055]FIGS. 5B and 5C illustrate alternate arrangements for zooming up;

[0056]FIG. 6 illustrates an example of intermediate stages of an imagewhen zooming down;

[0057]FIGS. 7A and 7B schematically illustrates the steps in the zoomingprocess according to a generalised embodiment;

[0058]FIG. 8 is a schematic illustration of six groups of twenty-twothumbnails, each group in a respective containment area, in which one ofthe containment areas is scaled and laid out in accordance with theaspect ratio zooming process of an embodiment;

[0059]FIG. 9 is a schematic illustration of the scaling of a group ofthumbnails of FIG. 8 in accordance with the aspect ratio zooming processaccording to an embodiment;

[0060] FIGS. 10 to 12 illustrate disadvantageous aspects of usingcontainment areas having different aspect ratios than that of thelargest displayable area;

[0061]FIG. 13 is a schematic illustration of the layout and scaling ofseveral groups of thumbnails of FIG. 8 in respective containment areashaving the same aspect ratio as the largest displayable area inaccordance with the process of an embodiment;

[0062]FIG. 14 is an illustrative example of four groups of thumbnailsorganised in respective containment areas, each having the same aspectratio as the largest displayable area, in accordance with the process ofthe an embodiment;

[0063]FIG. 15 illustrates aspect ratio zooming of one of the containmentareas of FIG. 14, which in turn comprises (sub) groups of thumbnailsorganised in (sub) containment areas having the same aspect ratio asboth the parent containment area and the largest displayable area;

[0064]FIG. 16 is a flow diagram illustrating the process according tothe preferred embodiment;

[0065] FIGS. 17A-17D illustrate the fast zoom process of FIGS. 4 to 7 asapplied to the containment areas of FIGS. 14 and 15;

[0066]FIG. 18 is a flow diagram illustrating the encoding of a thumbnailin accordance with another embodiment;

[0067]FIG. 19 is a flow diagram illustrating scaling of a thumbnail inaccordance with another embodiment;

[0068]FIGS. 20 and 21 depict the use in combination of the embodimentsof FIGS. 1 to 9 and 13 to 19.

[0069]FIG. 22 is a high-level block diagram illustrating the imagerepresentation technique according to the SWEET processes;

[0070]FIG. 23 is a diagram illustrating partitioning according to aSWEET process;

[0071]FIG. 24 is a flow diagram illustrating the method of representing,or encoding, an image according to a SWEET process;

[0072]FIG. 25 is a detailed flow diagram illustrating the step of codinga region in FIG. 24;

[0073]FIG. 26 is a flow diagram illustrating the method of decoding acoded representation of an image produced in accordance with the methodFIG. 24;

[0074]FIG. 27 is a detailed flow diagram illustrating the step ofdecoding a region in FIG. 26;

[0075]FIGS. 28A to 28D are diagrams illustrating the processing of atwo-dimensional, eight-coefficient region in accordance with theencoding and decoding method of FIGS. 24 to 26; and

[0076] FIGS. 29 to 32 are flow diagrams illustrating the methodrepresenting, or encoding, an image according to an alternate SWEETmethod.

DETAILED DESCRIPTION

[0077]FIGS. 1 and 2 illustrate a representative embodiment of such apersonal computer system 900 for implementing the processes to bedescribed. The computer system 900 includes a computer 901, and a videodisplay monitor 904. It may also include one or more input devices902,903. User inputs to operate the computer 901 are provided by theinput devices. For example, a user can provide input to the computer 901via a keyboard 902 and/or a pointing device such as the mouse 903. Thecomputer system 900 may be connected to one or more other computersusing a communication channel such as a modem communications path, acomputer network, or the Internet, for example. Further, any of severaltypes of output devices including plotters, printers, laser printers,and other reproduction devices may be connected to the computer module901.

[0078] The computer module 901 has one or more central processing units(CPU or processor) 905, memory 906 including random access memory (RAM),static RAM or cache, and read-only memory (ROM), a videointerface/adaptor 907, an input/output (I/O) interface 910, and storagedevice(s) generally depicted by block 909 in FIG. 9. A videointerface/adaptor 907 connects to the video display monitor 904 providesvideo signals from the computer module 901 for display on the videodisplay monitor 904. Optionally, an image or video capture device 920(eg., a digital camera) can be connected to the computer 901 via theinterface 910 as a source of digital images or video sequences that canbe represented by thumbnails in accordance with the describedembodiment.

[0079] The storage device(s) 909 may comprise a floppy disc, a hard discdrive, a magneto-optical disc drive, magnetic tape, CD-ROM and/or anyother of a number of non-volatile storage devices. The components 905 to910 shown in FIG. 9 are coupled to each other via a bus 911. The bus 911in turn can comprise data, address, and control buses. The overallstructure and individual components of the computer system 900 isessentially conventional and would be well known to persons skilled inthe art. Thus, the system 900 is simply provided for illustrativepurposes and other configurations can be employed without departing fromthe scope and spirit of the invention. The computer system may beimplemented using an IBM personal computer (PC) or compatible, one ofthe Macintosh™ family of PCs, Sun Sparcstation™, or the like. Theforegoing are merely exemplary of the types of computers with which theembodiments of the invention may be practiced.

[0080] In FIG. 3, representative components of the computer system 900are shown which include a hardware system 925, an operating system 930and application software 935. Typically, the hardware system 925includes the physical components of the computer system 900 describedabove with reference to FIGS. 1 and 2. The operating system 930traditionally includes management software applied within the computersystem 900 for performing various tasks and handling the operation ofthe hardware. The application software 935 includes specific routinesconfigured for performing user desired tasks. The operating system 930and application software 935 may reside in a permanent storage device909 or in the memory 906 or be temporarily loaded from the device 909 tothe memory storage 906.

[0081] As illustrated, the operating system 930 includes an input devicemanagement system 931 which receives user instigated control signalstypically from the keyboard 902, via a keyboard interface 926, and themouse 903 via a mouse interface 927. The input device management system931 processes the user's commands and transfers those to the applicationsoftware 935 being operated at the time, in this case being thumbnailmanagement software 936. As seen, the thumbnail management software 936includes an output to a file system 933 incorporated within theoperating system 930. The file system 933 provides for access to filesand folders which may be stored on a disk drive 929 accessed via a diskI/O interface 928 each within the hardware system. This drive 929 willbe appreciated can comprise a part of the storage device 909 shown inFIG. 2. Further, the thumbnail management software 936 can output imagecomponents to a drawing management system 932 which in turn provides forthe display of images on the display 904 via the video interface 907.Thus, for example, as shown in FIG. 1, the thumbnail management systemmay provide for the display of thumbnail images 940 which are includedas components as a interactive graphical user interface to a filemanagement system 942 operating within the computer system 900.

[0082] In this embodiment, the thumbnail management system incorporatesa number of components, one being a sizing component 937 which providesfor the zooming of images and in particular thumbnail images, a groupingarrangement 938 which provides for the coordination of groups ofthumbnail images, and a aspect ratio component 939 which provides forthe coordination of the aspect ratio of various thumbnail groups.

[0083] Fast Zooming of Images

[0084] In dealing with images in general and colour thumbnail images inparticular, it is desirable for those images to be zoomed-in andzoom-out rapidly when using file management systems withoutoverburdening the processor 905 with excessive graphical and objectand/or pixel-based image manipulations.

[0085] Referring to FIG. 4, there is shown a first image 10 as displayedfor example on the display device 904 at a first predetermined size andwhich is intended to be displayed as a second image 11 at a secondpredetermined size. In many instances, the first and second sizes arevariations (typically reductions) of the true image which, whencorrectly and fully displayed may occupy a full video display of say1024×768 pixels at 24 bits of RGB colour per pixel. This true image istraditionally stored on the disk storage device 909 in a compressedformat, for example using the JPEG standard. Typically, the second image11 corresponds to the first image, such as a thumbnail image, but atanother resolution scale. For example as illustrated, the first image 10is a 4×4 pixel image which is enlarged through at least one intermediatedisplayed size (not shown in FIG. 1) to display substantially the sameimage at a 16×16 pixel image resolution (ie: the image 11).

[0086] Often, a “Thumbnail” size image is referred to as being asmallest one of a number of predetermined sizes of a desired image. Thefirst image 10 of FIG. 4, for example, is referred to herein as the“Thumbnail” size image and the second image 11 is designated the desiredimage. Shown in FIG. 4 is a bi-directional arrow 12 representing that athumbnail size image 10 can be zoomed to a desired image 11 or a desiredimage can be zoomed to a thumbnail size image as hereinafter described.

[0087] A thumbnail size image 10 and a desired image (eg. 11) can bestored in a memory device in a compressed format (representation), andpreferably the thumbnail size image and the desired image are variousresolutions of a single compressed image. In various image compressionrepresentations, a multitude of resolutions can be extracted from asingle image stored on a memory storage device. As an example, an imagecompressed through wavelet coding allows an extraction of a plurality ofresolutions of the compressed image. A wavelet compressed image can bethought of as a bit stream of ones and zero. By applying a decompressionalgorithm to the entire bit stream, the wavelet compressed image issubstantially recovered (ie. wholly decompressed). However, by applyingthe decompression algorithm to a first portion of the bit stream, alower resolution version of the wavelet compressed image can berecovered and thus, by applying the decompression algorithm to a greaterportion than the first portion of the bit stream, a greater resolutionversion than the lower resolution version can also be obtained. Anexample of such a wavelet compression/decompression arrangement isdiscussed in detail later in this specification under the heading“SWEET—A Method For Digital Image Compression”. Other examples ofcompression techniques that can be adapted to extract a plurality ofresolutions of a single compressed image include JPEG (JointPhotographic Expert Group) compression.

[0088] Referring to FIG. 5A, there is shown a thumbnail size image 20which is extracted at a first resolution from a compressed true imageand displayed on a display device. The thumbnail size image 20 is to beenlarged to a desired image 21 at a predetermined size greater than thethumbnail size image 20. The thumbnail size image 20 is to be enlargedthrough a plurality of intermediate step sizes 22,23,24 to the finalpredetermined size of the desired image 21. In the present example, theintermediate sizes 22,23,24 are achieved by pixel replication. Eachpixel 25 of the thumbnail size image 20 is replicated (reproduced) fourtimes to obtain a block 26 of four pixels for a first intermediate size22 of the image 20. Each pixel of the block 26 of the first intermediatesize 22 is replicated four times to produce a block 27 of four pixels ofa second intermediate size 23 of the image 20. At this stage therefore,the original pixel 25 has been replicated to form sixteen pixels. Thisblock replication process is repeated until a predetermined final sizeis achieved. Preferably, the predetermined final size is the same size(dimensionally) as the desired image 21. In this connection it is notedthat the images 20-24 of FIG. 5A are not drawn to scale and the relativedepiction of the pixels are for illustrative purposes. At this stage thedesired image 21 is decompressed (extracted) from the compressed imageand is displayed at the predetermined final size of the image 20. Atleast one of the intermediate stage sizes 22,23,24 is preferablydisplayed on the display device to give a visual appearance of“zooming-in” from a thumbnail size image 20 to a predetermined finalsize of a desired image 21.

[0089] In this fashion, zooming-in on a thumbnail can be achieved bypixel replication and is preferably performed whilst decompression tothe desired size/resolution of the true image is taking place so thatonce instigated by the user, for example by clicking the mouse 903, thezoom “appears” to occur as the replicated stages are re-displayed. Oncompletion of the zoom however, a more accurate, non-replicated butdecompressed image is thus displayed. Thus, only one image (ie. thedesired final image) need be decompressed and so only one decompressiontime is required to achieve and display the zoom-in effect. During thattime, the user is presented with a progressively staged zoom of therudimentary thumbnail image which, whilst not of high image quality,provides the user with a transitory progression between image sizes thusassisting in maintaining user orientation within the file/folderstructure during browsing operations.

[0090] The pixel replication process described above is made withreference to a 2×2 (4 times) increase in size of the intermediate stagesizes. However, increasing the intermediate stage sizes at other thanmultiples of four can be optionally performed, as illustrated in FIGS.5B and 5C.

[0091] As an example of stretching the thumbnail image 20 by pixelreplication, at other than by a multiple of four, an interpolated valuebetween two adjacent pairs of pixels in the thumbnail image 20 can beobtained and the interpolated value is replicated three times to producethree output pixels for each dimension of a first intermediate size.This results in an appearance that the thumbnail size image 20 hasincreased in size by 1.5 for each dimension (1.5×1.5).

[0092] As seen in FIG. 5B, a 4×4 thumbnail image 28A is provided havingpixel values a,b,c, . . . p. In this example, it is desired to stretchthe thumbnail image 28A by a factor of 1.5 in each dimension to providea 6×6 intermediate stage. As noted above, adjacent pairs of pixels ofthe thumbnail image are interpolated. In this fashion, adjacent rowpairs of pixel values “a” and “b” are interpolated to provide a pixelvalue “1”, which is then replicated three times. Similarly, pixel values“c” and “d” are interpolated to give an interpolated value of “2” whichis also replicated three times. This process is applied to each adjacentpair of pixels in each row of the thumbnail image 28A to provide a 6×4array 28B of pixel values 1 . . . 8. In a similar fashion, but workingwith adjacent values in each column, interpolation proceeds. Pixelvalues 1 and 3 are interpolated to provide a pixel value 9 which isreplicated three times. This occurs for each of those pixels in thatcombination of values. Thus, a 9×9 array of pixels 28C is formed whichcomprises four interpolated values, 9, 10, 11 and 12 spread amongst 36pixel locations.

[0093] With reference to FIG. 5C, an alternative method can be providedwhereby all adjacent pixels are interpolated. As shown in FIG. 5C, aninput 4×4 array of pixels 29A comprising pixel values a, b, c . . . p isprovided. An interpolated value “1” is obtained from input values a andb and is replicated three times as illustrated. An interpolated value 2is obtained from adjacent pixels b and c and also replicated threetimes. Similarly, an interpolated value 3 is obtained from adjacentpixels c and d and also replicated. This process proceeds for each rowof the input array to reveal a 9×4 array of pixels of interpolatedvalues 29B. In a corresponding manner to the arrangement of FIG. 5B, thearray of pixels 29B is further interpolated based upon the adjacency ofthe values such that a value 13 is obtained by interpolating values 1and 4 and interpolated value 18 is obtained by interpolating the values4 and 7, and so on. Thus, it will be appreciated using this approachthat a 9×9 array 29C of pixels can be obtained from a 4×4 array, givingan enlargement factor of 2.25.

[0094] Further, whilst the foregoing has been described in relation tothe interpolation of adjacent and pairs pixels, other mathematicalprocesses may be applied to obtain the desired result. For example,adjacent pixel values may be averaged or blends provided betweenintermediate new pixels. However, it will be appreciated that thecomplexity of some mathematical operations will increase thecomputational time required to determine the intermediate zoomingstages. As the intermediate zooming stages are typically quitetransient, the need for higher quality is generally not necessary as allthat is required is for the user to be given the impression of aparticular thumbnail being enlarged by zooming-in. Thus, distortions ofthe image due to pixel replication/averaging etc. are tolerable as it isthe original image as decompressed which results and which leaves thegreatest lasting impression upon the eye of the user.

[0095] With reference to FIG. 6, a desired image 30 is to be contracted(shrunk in size) through a plurality of intermediate stage sizes31,32,33 to a thumbnail size image 34, to produce a visual effect of“zooming-out”. The “zooming-out” technique is achieved by decompressingthe true image, from a compressed image format, to provide a desiredimage 30 at a first image size (dimension), and a first resolution. Thenan average for a plurality of pixels 35 is obtained to produce anaverage pixel value, the average pixel value being assigned as a singlepixel of a first intermediate stage size 31 of the desired image 30. Inthe illustrated example of FIG. 3, each group of four pixels 35 of thedesired image 30 is averaged to produce a single pixel 36 of the firstintermediate stage 31. A group of four pixels 36 of the firstintermediate stage size 31 of the desired image 30 is averaged toproduce a single pixel 37 of a second intermediate stage size 32. Thisprocess is repeated for each four pixel grouping of the firstintermediate stage 31 until each pixel of the second intermediate stagesize 32 of the desired image 30 is produced. The process of averagingfour pixels to produce a pixel of a next intermediate stage size 33 isrepeated until preferably a thumbnail size 34 of the desired image 30 isreached. Once the thumbnail size 34 image is substantially achieved, alow resolution image may be extracted from the compressed image anddisplayed at the thumbnail size 34. Again, as with the pixel replicationprocess described with reference to FIGS. 5B and 5C, the desired imagecan be reduced in size by other than a factor of four. Averaging, forexample, twenty-five pixels of one intermediate stage size to achievefour pixels of a next intermediate stage size, results in a reductionsize ratio of 6.25:1.

[0096] An alternative option for shrinking the size of an image is tosub-sample the image. That is, one of every n number of pixels of animage is selected as a pixel to form a reduced image. The reduced imagehaving a reduction (size) ratio n:1.

[0097] Referring now to FIG. 7A, a schematic flow diagram illustratessteps for zooming-in or zooming-out an image in accordance with theembodiments of the present invention, which are preferably implementedas software operating within the sizing module 937 of the applicationsoftware 935.

[0098] As seen in FIG. 7A, commencing with a start step 46A, a firstimage is extracted at step 40 (decompressed) from a compressed inputimage and displayed on a display device at a first size and resolution.A stretch (or shrink) 41 process is applied, according to at least oneof the techniques described above, and a first intermediate size of thefirst image is obtained. The first image is cleared from the display 904and the first intermediate size of the first image is displayed.

[0099] A checking process 42 determines if the first intermediate sizeis dimensionally a size substantially similar to a predetermined imagesize. If the checking process determines that the predetermined size hasnot been reached, the stretch (or shrink) process 41 is again applied 43to the current (first intermediate) image to result in a secondintermediate stage size of the first image which is displayed in placeof the first predetermined size. The process is continued until thechecking procedure 42 determines that the predetermined image size hasbeen, in substance, reached at step 44. At this stage, the procedure maystop, resulting in the continued display of the stretched/shrank imageat the desired size. Preferably however, a second image being a secondsize and resolution of the first image is decompressed (extracted) fromthe compressed input image and displayed on the display device. Thestretch process 41 is used when zooming-in, and the shrink process 41 isused when zooming-out. It follows that the stretch or shrink process 41cannot be used alternatively to the extent that such would produce anabsurd outcome in either a zooming-in or zooming-out of an image.

[0100] In the embodiment of FIG. 7B, a similar process to that of FIG.7A is illustrated but where a start step 46B instigates both step 40 todecompress and display the input image at the first size, and also step45B to commence the decompression of the input image. In this fashionboth decompression and low quality stretching/shrinking occursimultaneously and the second image is displayed after the second imagesize of the stretched/shrunk image is obtained and decompression hasbeen finished, as assessed by decision step 48.

[0101] In each of the examples of FIGS. 7A and 7B, the input image neednot be a fully compressed true image but may comprise a partlycompressed image or indeed a utility version thereof such as a thumbnailor any of the intermediate stages described above.

[0102] It will be apparent from the above that zooming, both in and out,of images can be performed in such a manner to display a stagedprogression of the zoom process thus aiding user orientation, butwithout the need for a complex compression to occur at each displaystage. The stretching/shrinking of the intermediate (and possibly final)stages occupies substantially fewer computational resources within thecomputer system 900 and thus can be performed faster than incrementaldecompression. Thus image browsing operations, particularly in filemanagement systems are facilitated and enhanced by speed and ease ofuse, whilst maintaining the ability for full image reproduction.

[0103] Aspect Ratio Zooming of Thumbnails

[0104] This embodiment is directed to a method, apparatus, system andcomputer program product for aspect-ratio zooming of thumbnails used forimage browsing using a computer. This essentially separates the tasks ofbrowsing content and assessing content, thereby eliminating the need tocompromise on image thumbnail size. The process of aspect-ratio zoomingrepresents a new concept in image browsing and provides a uniquesolution to the problems of navigation and orientation in vast imagespaces.

[0105] Using the aspect ratio zooming process of the preferredembodiment, all thumbnails within the search domain are scaled to fitwithin the available display area. This facilitates the global browsingof content. Thumbnails of interest are then scaled up as required. Thisfacilitates more detailed assessment of image content.

[0106] The aspect ratio zooming process of the preferred embodimentenables implementation of a unique image browsing system that provides aconsistent spatial representation of the entire search domain, supportfor image groupings within a hierarchical, user defined structure, andrapid traversal between browsing content and detailed assessment ofcontent. An important aspect of this aspect ratio zooming process is theprocess of recursively laying out image display areas at fixed aspectratios, where aspect ratio is defined as the ratio of the width of theimage display area to the height of the image display area. Maintaininga fixed aspect ratio for image display areas overcomes the problems ofuser navigation and orientation within the entire image space, as wellas the technological problems associated with scaling images up and downat adequate speed so as not to inhibit the browsing process.

[0107] The apparatus for aspect-ratio zooming of thumbnails used forimage browsing in accordance with this method is preferably implementedusing software executing on a personal computer, such as the systemshown in FIGS. 1, 2 and 3 for example. In particular, the softwarerunning on the computer lays out one or more groups of thumbnails of thedigital images in containment areas having a predetermined aspect ratiocorresponding to that of the largest possible available display area ofa display device, and scaling the thumbnails, preferably as large aspossible, to fit within the respective containment area.

[0108] Preferably, the apparatus is implemented with the aspect ratiozooming process embodied as software or a computer program (eg, within abrowser application) recorded on a computer readable medium and loadedinto the computer system 900 from the computer readable medium. Examplesof suitable computer readable media include a floppy disk, magnetictape, a hard disc drive, a ROM or integrated circuit, a magneto-opticaldisk, a radio or infra-red transmission channel between the computer andanother device, a computer readable card such as a PCMCIA card, anetwork connection to another computer or networked device, and theInternet and Intranets including e-mail transmissions and informationrecorded on websites and the like. The foregoing are merely exemplary ofrelevant computer readable media Other computer readable mediums may bepracticed without departing from the scope and spirit of the invention.

[0109]FIG. 16 is a flow diagram illustrating embodiments of the aspectratio zooming process 120 preferably implemented by the software module939 of FIG. 3. Processing commences at step 121 in which the thumbnails,organised in one or more groups, are laid out in respective containmentareas. Each containment area may in turn contain (sub) containment areasand has a predetermined aspect ratio. The aspect ratio is the ratio ofthe width to the height of the largest displayable area available (whichmay be the entire screen, a portion of the screen, or a window, forexample) on the display device. This is described hereinafter in greaterdetail with reference to the FIGS. 8 to 15. In step 123, the thumbnailsare scaled in each containment area to fit with the available area ofthe containment area. It will be appreciated that this forms ahierarchical, tree-like structure of sorts, in which one containmentarea may form the uppermost containment area within the displayablearea. Steps 121 and 123 constitute the essential steps of thisembodiment. The process may constitute these steps alone, but optionallyalso include steps 124 to 126.

[0110] Preferably, a check is made in decision block 124 to determine ifa user selected containment area is to be zoomed in. If decision block124 returns true (zoom-in), processing continues at step 125. In step125, the selected containment area is enlarged (zoomed up) to theavailable display area, and this is likewise carried out for any(sub)containment areas with respect to the relevant parent containmentareas. Execution then continues at step 121, where the aspect ratiozooming process is applied in steps 121 and 123.

[0111] Otherwise, if decision block 124 returns false (zoom-out),processing continues at step 126. In this step, the reverse orcomplementary process of step 125 is carried before processing returnsto step 121. That is, a parent containment area of the selected group isset to the available display area, and likewise the selected containmentarea is reduced in size, along with its thumbnails.

[0112] The process of aspect ratio zooming-in this embodiment overcomesproblems of user navigation and orientation within the image space. FIG.8 shows an example of an entire image space containing a total of 132image thumbnails divided into six distinct groups. The image thumbnailsare represented by grey numbered squares. A primary containment area 100represents the entire space available for the display of imagethumbnails, for example, on a video display monitor. In general, theprimary containment area defines the aspect ratio of the display.Containment areas 102, 104, 106, 108, 110 and 112 provide visualsegregation for the six groups of images within the image space. Forthis example, each group contains 22 image thumbnails and displayed atuniform size and scaled to the largest possible size that permits thegroup to fit within the respective containment area. As seen in FIG. 8,containment area 108 is the only area that has the same aspect ratio asthe primary containment area 100, in this case a ratio of about 1.320.

[0113] During normal browsing or searching activity, users typicallyscan the entire search domain (in this case, the entire image space) andthen refine their search by choosing to take a closer look at perceivedareas of interest. In this case, the user may choose to zoom-in on theimages displayed in one of the containment areas. Containment area 108as illustrated has been laid out in accordance with the process of thepreferred embodiment. Only the images in containment area 108 (which hasthe same aspect ratio as the primary containment area 100) can be zoomedup to fill the primary containment area 100 (ie., the entire spaceavailable for the display of images) without causing orientationproblems for users.

[0114]FIG. 9 illustrates the process of aspect ratio zooming thethumbnails of containment area 108, having the same aspect ratio as thelargest displayable area, in accordance with the process of thepreferred embodiment. Thus, the images in containment area 108 can beenlarged to fill the primary containment area 100 without beingrearranged. This is important since it provides a consistent spatialrepresentation of the image space. In contrast, zooming up one of theother image containment areas 102, 104, 106, 110 or 112 of FIG. 8 (sothat the image thumbnails within that group are made as large aspossible to fill the primary containment area 100) results in the imagesbeing rearranged as is described hereinafter with reference to FIG. 10.

[0115]FIG. 10 shows how the images in containment area 106 have to berearranged if they are zoomed up to fill the primary containment area100. Alternative methods of zooming up a containment area with adifferent aspect ratio to the primary containment area 100 are shown inFIGS. 11 and 12. Both of these necessitate the use of scroll bars114,115 (or similar controls) and lead to orientation and navigationalproblems for users. The useability problems associated with using scrollbars are known.

[0116]FIG. 13 shows how the thumbnails of FIG. 8 can be rearranged inaccordance with the aspect ratio zooming process of the preferredembodiment. Each of the containment areas 102 to 112 of the sixthumbnail groupings is laid out to have the same aspect ratio as theprincipal containment area 100. In this manner, all six image groupingsare displayed in areas of the same aspect ratio as the primarycontainment area.

[0117]FIG. 14 illustrates an example of the aspect ratio zooming processextended so that image containment areas with fixed aspect ratios candisplay other image containment areas with the same aspect ratio. Thecontainment areas containing these groupings may also in turn comprise(sub) containment areas. Thus, a hierarchical structure of containmentareas and (sub) containment areas is provided. The structure of theimage space is depicted in the (Windows or Internet) “Explorer”™ typetree view 130 on the left hand side of the screenshot or image. Each ofthe containment areas is depicted as a folder in the tree. The largestdisplayable area or principal containment area is labelled “Library”132, and it contains four groupings of thumbnails labelled “Australia”134, “Transport” 136, “Images of the World” 138, and “People” 140.

[0118] Whereas the containment area 134 labelled “Australia” containsonly image thumbnails, the containment area 140 for “People” containsfurther image thumbnail groupings (“Children” 141, “Corporate people”142, “Couples” 143, “Sports people” 144). This equally applies to thecontainment areas “Transport” and “Images of the World”. In this way,the aspect ratio zooming supports the meaningful and concurrent displayof an entire image space.

[0119]FIG. 15 illustrates the enlargement of the “People” containmentarea 140 so as to fill the available display space (formerly labelled“Library”). Having had a closer look at the contents of the “People”group, a user can choose to “zoom-in” further on another containmentarea (for example, “Sports people” 144 contains four further groups145-148), or zoom back out. In this way, users are able to reduce orenlarge any containment area to fill the display space available whilemaintaining context and spatial reference.

[0120] The process of aspect ratio zooming according to the preferredembodiment is advantageous in that it overcomes disadvantages of theprior art. This includes the technological problems associated withscaling images up and down at adequate speed. Zooming (moving fromdisplaying a group of images and/or containment areas, to displaying thecontents of one containment area) can be performed in a number of ways:

[0121] 1) Switching displays: simply switch from the initial state tothe zoomed in state, which is advantageous in that it is very fast;

[0122] 2) Zooming with outlines: drawing the outline of each image orcontainment area being zoomed at each zoom step, which is also veryfast, and gives the user some feel for the zoom process;

[0123] 3) Accurate zooming: performing a number of intermediate zoomsteps, and redrawing the display accurately at each step, which isadvantageous in that each zoom step looks appealing;

[0124] 4) Bitmap scaling: scaling up (or down) a bitmap of the whole (orpart of) the primary containment area 100, to each zoom step, and thenaccurately redrawing at the end of the zoom; this is very fast and givesthe user a good feel for the zoom. “Scaling the bitmap” in above item(4) is easily performed if the bitmap of the area being zoomed into hasthe same aspect ratio as the viewing area. If it does not have the sameaspect ratio, the zoom steps do not correspond with the final resultingdisplay. An example of such zooming is the fast zoom described abovewith reference to FIGS. 4 to 7B.

[0125] After the last zoom step, the display contains the “scaledbitmap”. By drawing the background quickly, but leaving the scaledimages untouched, a good approximation of the final display may beobtained. Each image may be redrawn properly one at a time whileallowing the user to interact with the program. The longer the userwaits, the better is the quality of the display (until all the imagesare redrawn).

[0126] A further and preferred implementation is illustrated in FIGS.17A-17D which combines aspect ratio zooming with an alternativeimplementation of the previously described fast zooming of thumbnails.In particular, the present inventors have determined that image-by-imagethumbnail stretching as may be required when zooming a containment areacan be excessively time consuming particularly with the need to redrawthe background of the containment area with each intermediate stage.This difficulty is overcome by selecting the entire containment area(window) and processing that area as if it were a single input imagewhereby the entire containment area is stretched/shrunk including textand images, and when the final size is obtained using the fast zoomingprocess previously described, the entire containment area/window isredrawn including background, title and individual images. This isdepicted in FIGS. 17A-17D where it will be appreciated that FIG. 17A isa schematic representation of the library window 132 seen previously inFIG. 14. In this embodiment, it is desired to zoom-in upon the “people”containment area 140 to reveal the image shown in FIG. 15. FIG. 17Ashows the library display in its initial form prior to the commencementof the zoom. In FIG. 17B, the people containment area 140 has beenzoomed to a first intermediate size and is shown illustrated commencingto occlude the other containment areas. FIG. 17C shows a furthertransitional stage where the people containment area 140 substantiallyoccludes the remainder as the zoom proceeds. FIG. 17D shows theconfiguration of the display at the completion of the zoom. In theintermediate displays of FIG. 17B and FIG. 17C the people containmentarea 140 is displayed using pixels replicated from the peoplecontainment area shown in FIG. 17A. In FIG. 17D however, each of theindividual thumbnails and sub-containment areas 141-144 are decompressedto reveal the desired higher quality images.

[0127] The process of aspect ratio zooming according to the preferredembodiment is also advantageous in that it provides a new and superiorimage browsing system given accepted wisdom regarding databasenavigation and the principles of browsing. The process also complieswith the guiding principle for the design of database navigationalsystems, which is to make users aware of the entire contents of thesearch domain and to facilitate rapid examination of any individual itemin greater detail (eg, see Spence, R. and Apperly, M. D. (1982)“Database Navigation: An Office Environment For the Professional”,Behaviour and Information Technology, 1, 1, pp 43-54).

[0128] The process also exploits the remarkable (and well documented)spatial memory and visual scanning capabilities of humans to assist inthe browsing process (rather than simply relying on recall through priorclassification). Still further, it is advantageous in that it providescontinual reinforcement as to the structure and layout of the imagespace which, albeit unconscious, is generally considered to enhancespatial memory. It also capitalises on the almost limitless capacity ofhuman recognition memory (eg, see Standing, L. (1973) “Learning 10,000Pictures”, Quarterly Journal of Experimental Psychology, 25, pp.207-222) and the highly developed skills for pattern recognition bydisplaying all thumbnails in the search domain at maximum size possiblewhile maintaining context and providing consistent spatial reference.

[0129] While users may not always be happy making judgments about thecontent of an image from a thumbnail smaller than 80×80 pixels, imagethumbnails as small as small as 6×6 pixels can nonetheless be easilydistinguished from each other. Further, in combination withforeknowledge of the image content, the human perceptual system is ableto recognise images from thumbnails displayed as small as 14×14 pixels.Consequently, for the purposes of browsing (as distinct from assessingcontent) considerable value can be achieved by displaying thumbnails atmuch smaller sizes than they are displayed using prior art techniques.

[0130] Compressing and Scaling Thumbnails

[0131] This embodiment is directed to a method, apparatus, and computerprogram product for encoding and scaling thumbnails. That is, imagethumbnails are generated at varying resolutions or sizes. The variousresolutions or sizes may be predetermined, which can be adjusted to adifferent size than that of the predetermined one. Thumbnails providedin accordance with the preferred embodiment are particularly useful inapplications for image browsers used to navigate large databases ofdigital images. While the use of thumbnails produced in accordance withthe preferred embodiment has particular application to browsing imagedatabases, it will be apparent to those skilled in the art that such atechnique can be advantageously practiced in other applications withoutdeparting from the scope and spirit of the invention. In the followingdescription, numerous specific details, such as the size of imagethumbnails, application software embodying the process, screen displays,arrangements of thumbnails, etc. are described in detail to provide amore thorough description of the present invention. It will be apparent,however, to one skilled in the art that the present invention may bepracticed without these specific details. In other instances, well-knownfeatures are not described in detail so as not to unnecessarily obscurethe present invention.

[0132] Overview

[0133] The process of compressing and scaling a thumbnail in accordancewith the preferred embodiment enables the display of many smallthumbnails or image icons on the one hand, when necessary, and allowssuch thumbnails to also be displayed as large as possible on the otherhand. In a browsing application where elements of a database arerepresented by thumbnails, the encoding and scaling process can beadvantageously employed To optimally meet the noted trade-off,thumbnails preferably fill a display window when displayed and do notunnecessarily leave significant portions of a display window unused. Asa rough approximation, the thumbnails fill a constant “image data”regardless of the number of thumbnails displayed. If there are manyimages to be displayed, the thumbnails are made small, while if thereare only a few thumbnails to be displayed they are made larger. The ideaof a constant image area is only approximate as the amount of areadepends on the size of the border areas which change with image iconsize and number.

[0134] The process of the preferred embodiment also provides compressionto minimise storage requirements for such thumbnails. Even small imagesrequire a significant amount of space and hence a large number ofthumbnails can use a significant amount of memory. Compression is alsodesired for memory efficiency. It is often desirable that manythumbnails are stored in memory, rather than on disk, for faster access.Memory is normally at a premium and therefore the process of thepreferred embodiment stores the thumbnails in memory in a compressedformat.

[0135] In this process, a hierarchical image icon representation is usedto meet the “constant area”/variable image size requirement. To usecompression, relatively fast decompression is required so that the imageicons are displayed in an acceptable amount of time. To display a“constant area” of images in substantially the same amount of time, theprocess of the preferred embodiment provides a decompression time thatis substantially proportional to the size of the image. By displayingthe different types of “constant area” image sets in the same amount oftime, the process ensures that each image set is displayed in anacceptable amount of time. If each image set is displayed insubstantially the same amount of time, a “worst case” display timeproblem can be minimised or eliminated using this process. Thus, allimage sets are displayed as quickly as possible.

[0136] To meet the foregoing requirements, the process of the preferredembodiment utilises an efficient hierarchical discrete wavelet transform(DWT) coded representation. DWT compression techniques offer good imagecompression, and the coding of the subbands produced by the DWT in termsof increasing frequency offers a hierarchical representation withoutsubstantially sacrificing compression efficiency (ie., amount ofcompression). Still further, coding of the subbands in such an orderoffers decompression times proportional to the size of the decodedimage, and independent of the size of the original image. Using asuitable DWT compressed representation also has another advantage. Theimage icons can actually be decompressed at a size greater than the sizeof the original compressed image icon. This is simply achieved using theinverse DWT engine within the decompression code. The process of thepreferred embodiment utilises the SWEET compression technique which isparticularly suitable for this compression application. The SWEETtechnique is disclosed in Australian Provisional Application No. PO4728filed on Jan. 22, 1997 and entitled “A Method for Digital ImageCompression”, the relevant disclosure of which is reproduced below undera corresponding sub-heading. Compression and decompression using SWEETare both very fast. Also, the subbands are coded in order of increasingfrequency and independently. Further, SWEET provides high compressionefficiency.

[0137] Because the process of the preferred embodiment uses a discretewavelet transform coding, it can decode small resolution versions ofimages with a decoding time linearly dependent on the size of thedecoded image. Therefore, a constant image “area” can be decodedindependent of the size of the images that constitute an area. Anefficient hierarchical DWT representation meets the desired requirementsof fast decompression and hierarchical size decompression withdecompression speed substantially proportional to the size of thedecoded image. Further, this hierarchical representation can be achievedwithout sacrificing compression efficiency and in fact compression canbe obtained competitive with state of the art techniques. Finally, usinga DWT representation, the inverse DWT engine can be used to display theimage icons at a size larger than the original compressed size.

[0138] Exemplary Environment Using Thumbnails

[0139] As discussed above FIG. 13 is a block diagram illustrating sixgroups 102-112 of thumbnails displayed on a display device 100 (eg., acomputer display monitor). Each of the exemplary groups 102-112 contains22 thumbnails numbered accordingly. While particular groupings andnumbers of thumbnails are shown, the embodiments of the invention arenot so limited, and the example of FIG. 13 is provided for ease ofdescription with each thumbnail depicted as a grey-filled block.

[0140] As discussed above, each thumbnail is a visual representation,preferably of a corresponding image file. In the case of image files,each image file has a reference to it as well as a correspondingthumbnail. The thumbnails or image icons provide an indication of theimage they represent, and are small versions of the images. Referencesto the image files are stored in a database (although this could easilybe extended to non-image files), and may be acquired from a file system,or from an input device such as a digital camera or a scanner. When animage file is added to the database system, a reference to the imagefile is stored, and a thumbnail is created for the image. While thepreferred embodiment uses thumbnails that represent still images, theinvention is not limited to this particular application. Instead,thumbnails using the method of the preferred embodiment described belowcan be used to represent other things including video sequences, adocument for which an icon can be made, etc.

[0141] As shown in the screenshot of FIG. 14, each of the thumbnails isa graphical representation of an image. Dependent upon the particularapplication involved, the arrangement of the thumbnails can reflect anumber of things: the semi-hierarchical categorisation of the imagesinto folders (a directed acyclic graph with a single root, but only theleaves, ie., the images themselves, can be multiply referenced); adisplay of the images brought into the system but not yet categorised;or the results of some automated search through the images in thesystem. Large numbers of images may be stored in the system. Forexample, image databases may not uncommonly contain ten thousand images.In fact the only limitation on the size of the image database is thestorage capacity and performance of the computer being used.

[0142] In this example, the top-level display logically shows all thecategorised images. For each internal node of the graph (each folder)shown on the left-hand side of FIG. 14, a rectangular array of thethumbnails is displayed in the top left corner and at the same size forall the leaves directly connected to this node. In the remaining space,an outline is laid out of every internal node that depends from thisnode. This process is repeated recursively (although at smaller scales)within each outline, ie. for each internal node. The hierarchy ofthumbnails is navigated by zooming-into and out of the folders in theexemplary application. Because a directed acyclic graph is beingrepresented, a given image may also appear several times in thehierarchy. Since the places where the image appears may be at differentlevels of the hierarchy, several thumbnails may simultaneously need tobe displayed for the image at different sizes.

[0143] Encoding and Scaling Process of Preferred Embodiment

[0144] To reduce the memory requirements for potentially large numbersof thumbnails and to provide rapid scaling of thumbnails, the preferredembodiment as depicted in FIG. 18 provides a method 150 of encodingthumbnails, where images are compressed and the form of compressionenables rapid scaling of the thumbnail using a hierarchicalrepresentation. Preferably, in step 154, one or more thumbnails aregenerated by compressing respective images using a hierarchical discretewavelet transform (DWT) compression technique. More preferably, thehierarchical DWT compression technique employed is the SWEET technique.

[0145]FIG. 19 is a flow diagram illustrating the process of scaling athumbnail in accordance with the preferred embodiment. In particular,the flow diagram illustrates the process 160 of decoding thumbnails,where each thumbnail comprises a hierarchical representation of adigital image, at a predetermined one of a plurality of scales toprovide the thumbnail for display having a desired size. Processingcommences in step 162. In step 164, a thumbnail comprising thehierarchical representation of an image is decoded for the purpose ofproducing a desired size or scale thumbnail. The thumbnail is decoded tothe scale closest to the desired size of the thumbnail for display. Thescale of the decoded hierarchical representation may be equal to thedesired size of the thumbnail. If it is not, the decoded hierarchicalrepresentation is preferably larger than the desired size. Again, thehierarchical representation is preferably an image compressed using theSWEET compression technique. In decision block 166, a check is made todetermine if the scale of the decoded thumbnail equals the desired sizeto be displayed. If decision block 166 returns true (yes), processingcontinues at step 170. Otherwise, if decision block 166 returns false(no), processing continues at step 168.

[0146] In step 168, the size of the decoded hierarchical representationis adjusted to the desired size of the thumbnail to be displayed.Preferably, the adjustment step involves uniformly scaling down ordownsampling the decoded hierarchical representation to provide thedesired-size thumbnail if the scale of the decoded hierarchicalrepresentation is greater than the desired size of the thumbnail.Scaling of the thumbnail is preferably always down, and by less than afactor of 2 in each linear dimension. In operating environments such asMicrosoft™ Windows™, in-built Windows functions may be used to do this,giving the system the opportunity to optimise the operation as best itcan.

[0147] In step 170, the decoded hierarchical representation is displayedas the thumbnail at the desired size. Coded and/or decoded hierarchicalrepresentations (eg, compressed and/or decompressed thumbnails) arepreferably stored in a cache or memory of thumbnails for later rapidretrieval and display. Optionally, because the process of decompressingthe hierarchical representation to the desired scale can involvedecoding several intermediate scales or sizes of the hierarchicalrepresentation, the intermediate scales may also be stored in the cacheor memory of thumbnails. For example, a decoded thumbnail displayed at asize of 64×64 pixels may subsequently be displayed at a size of 16×16pixels. If so, an earlier decoded thumbnail of this size may beretrieved from the cache, where the smaller size thumbnail was obtainedin the process of decoding the 64×64 pixel thumbnail. Processingterminates in step 172.

[0148] While steps 164 and 168 preferably involve downsampling to adjusta (larger size) decoded hierarchical representation to the desired size,other techniques of doing so may be employed without departing from thescope of the invention. For example, the method of FIG. 19 couldalternatively involve decoding the hierarchical representation to ascale smaller than the desired thumbnail size and upsampling the decodedhierarchical representation to provide the desired-size thumbnail. As afurther alternative, the method may involve decoding the hierarchicalrepresentation in step 164 at two different adjacent scales or sizes(eg, 64×64 pixels and 32×32 pixels) to provide the desired sizethumbnail. In step 168, interpolation can be carried out between thedecoded hierarchical representations to provide the desired-sizethumbnail. In this case, one of the scales of the decoded hierarchicalrepresentation is greater than the desired thumbnail size and the otherscale is less than the desired thumbnail size.

[0149] Returning to FIG. 15 is a further exemplary screenshotillustrating the zoomed up display of thumbnails or image icons of thePeople group of FIG. 14. The scaled up versions of the thumbnails shownin FIG. 15 may be obtained in accordance with the process of FIG. 19.

[0150] Preferably, the process described above is implemented with thethumbnail encoding and scaling process embodied as software or acomputer program (eg, within a browser application) recorded on acomputer readable medium and loaded into the computer system 900 fromthe computer readable medium. Examples of such computer readable mediuminclude a floppy disk, magnetic tape, a hard disc drive, a ROM orintegrated circuit, a magneto-optical disk, a radio or infra-redtransmission channel between the computer and another device, a computerreadable card such as a PCMCIA card, a network connection to anothercomputer or networked device, and the Internet and Intranets includinge-mail transmissions and information recorded on websites and the like.The foregoing are merely exemplary of relevant computer readable media.Other computer readable media may be practiced without departing fromthe scope and spirit of the invention.

[0151] The method according to the preferred embodiment, and inparticular using the SWEET compression technique, satisfies a number ofconcurrent requirements for storing and scaling thumbnails and thereforehas a number of advantages. Firstly, the compression of the thumbnailsallows as many thumbnails as possible to be stored in memory. This isparticularly important where large numbers of thumbnails may be loadedinto memory in an application where a user is for example browsing alarge database of images. More preferably, a cache may be used to storethumbnails for rapid access, including multiple, different-sized decodedversions of thumbnails. Secondly, the preferred embodiment makes itpossible to decompress thumbnails quickly. This provides rapid responsetime for users, especially in applications involving navigation of adatabase, such as an image database, where a user may zoom a thumbnailup or down. The method according to the preferred embodiment providescompression efficiency and decompression speed. Thirdly, the preferredembodiment provides satisfactory image quality at all sizes or scales atwhich thumbnails may be displayed. This third requirement is achieved inthe preferred embodiment without requiring multiple compressions of thethumbnails stored at varying sizes as might be required by JPEGcompression, or having a single compression of an image at the largestpossible size needed and then scaling down uniformly. Thus, thepreferred embodiment does not wastefully use up more storage and memory,and is not slow.

[0152] Compressing an image to produce a hierarchical representationusing the SWEET compression technique has a number of advantages. Thecompression achieved by SWEET is typically 10:1 and therefore provides agood compression factor. Further, decompression using SWEET is fast andlinear in relation to the number of pixels decompressed. Thus, thedecompression time for displaying a large number (eg, a screen full) ofsmall thumbnails is of the same order as displaying a few large sizethumbnails. Further, SWEET hierarchically encodes thumbnails, andtherefore it is not necessary to decompress the whole data set to get atthe smaller size thumbnails. In fact, because the SWEET compressiontechnique sequentially codes data, the first portion of SWEET compresseddata contains all the information for decompressing the smallest sizeimage, the next portion contains additional information required fordecompressing the next larger size, and so on. Still further,performance is further enhanced because in the case of displaying anumber of the smallest size thumbnails, all the data can be prefetchedby the CPU cache.

[0153] With regard to step 168 of FIG. 19, full use of the highestavailable resolution stored by SWEET is made. In particular, SWEETprovides a “minus 1” decompression, which provides a thumbnail which istwice the size (in each linear dimension) of the uncompressed thumbnail.When the minus 1 image is scaled down, satisfactory image quality isstill provided. If a thumbnail is desired for viewing at a size that islarger than the maximum size provided by SWEET (which is a tunableparameter), the “minus 1” version can be scaled up. A further advantageof the preferred embodiment is that system can handle thumbnails whoseuncompressed forms have different sizes.

[0154] As will be understood from the foregoing the, method according tothe preferred embodiments permits thumbnails or images to be displayedin various different arrangements that require, at least, one or more ofthe following: one or more different images; or one or more images atdifferent uniform scales; or one or more images in different positions.The method also facilitates changing between different displayarrangements quickly. Images can be displayed with satisfactory qualityregardless of the size.

[0155]FIG. 20 illustrates in flowchart form how the various embodimentsdescribed above may be practically implemented in the computer system900. At step 181 the aspect ratio of the display 904 is identified frompredetermined hardware parameters. At step 182, the containment areawithin the browser application software is adjusted in shape to have acorresponding aspect ratio as described with reference to FIGS. 8 and13. At step 183 the number of thumbnails are identified so as to permitdetermination of the thumbnail size within the containment area. At step184 the thumbnail size is adjusted according to FIGS. 4-7B, 16 or 18-19so that all thumbnails fit within the containment area. At step 185, there-sized thumbnails are arranged in the containment area and finally atstep 156 the user may re-size the containment area to facilitate ease ofbrowsing.

[0156]FIG. 21 shows how the above appears at a user level. Steps187A-187C each identify directory movements that may be instigated bythe user in a conventional manner. When the correct directory isidentified at step 187B, at step 188A the user views the directory toidentify a target image. Such an image is selected at step 188B whichcauses a fast zoom-in as described above. The user at step 188C views anintermediate sized image to confirm the correct image. If not, azoom-out is performed and another selection made at step 188A. If thecorrect image is identified at step 188C, that may be selected to causethe true image to be decompressed and displayed at a high quality level.In each of the directory operations 187A-187C, the aspect ratio basedmanipulation of directories and containment areas may be performed,which in the preferred embodiment is combined with hierarchicalrepresentation of the image and its zooming.

[0157] SWEET—A Method for Digital Image Compression

[0158] This disclosure relates to a method and apparatus forrepresenting digital image data, and in particular to a method andapparatus for encoding and decoding transform coefficients obtained fromdigital image data.

[0159] A number of transform-based image coding techniques are knownwhich involve linear transforming a source image to decorrelate data andthen encoding the transform coefficients. Such conventional techniquesinclude the JPEG standard image compression method, which employs an 8×8block discrete cosine transform (DCT). JPEG encoding involvestransforming blocks of a source image using the DCT, quantising theresultant transform coefficients where most of the compression iseffected taking advantage of visual perception, and lossless encodingthe quantised coefficients in a predefined zig-zag sequence from lowestfrequency coefficients to highest frequency coefficients.

[0160] There is also a compression technique termed the embeddedzerotree wavelet (EZW) method. EZW involves applying a discrete wavelettransform to a source image to decompose the image into a number of highfrequency subbands and a lowest frequency subband, normally at a numberof different resolution levels or scales. Zero tree encoding is thenapplied to the subbands dependent upon predictions of theself-similarity of coefficients across scales. The zero-tree-encodedcoefficients are then lossless encoded using arithmetic coding.

[0161] However, both techniques utilise complex methods for encodingposition information and employ lossless encoding. Thus, the foregoingmethods have a number of disadvantages including lack of flexibility andcomplexity in the coding technique.

[0162] In essence, the SWEET process of the present disclosure providesa method of representing a digital image to provide a codedrepresentation, the method comprising the steps of:

[0163] transforming the digital image to derive a plurality ofcoefficients, each coefficient represented by a predefined bit sequence;

[0164] selecting a portion of the plurality of coefficients as a region;and

[0165] (a) scanning the significance of each bitplane of the selectedregion from a most significant bitplane towards a least significantbitplane and providing a first token in the coded representation foreach insignificant bitplane until a significant bitplane is determined,wherein a second token is provided in the coded representation for thesignificant bitplane;

[0166] (b) partitioning the selected region into two or more subregionshaving a predetermined form, and setting each of the subregions as theselected region; and

[0167] (c) repeating steps (a) and (b) commencing from the significantbitplane until the selected region has a predetermined size, wherein thecoefficients of the selected region are coded and provided in the codedrepresentation.

[0168] Preferably, the linear transforming step comprises applying adiscrete wavelet transform to the digital image, and the selectedportion of the coefficients initially comprises the entire plurality ofcoefficients, or a subband of the coefficients.

[0169] Preferably, the first and second tokens comprise bit values of 0and 1, respectively.

[0170] Preferably, the subregions are equally sized, and optionally aresquare.

[0171] Preferably, the predetermined size of the subregion is a 1×1coefficient. Still further, the 1×1 coefficient is encoded by outputtingbits of the corresponding bit sequence beginning with the respectivesignificant bitplane. Preferably, only bits of the corresponding bitsequence above a predetermined minimum bit level are output in the codedrepresentation.

[0172] Preferably, in step (c), steps (a) and (b) are repeated otherwiseuntil each bitplane of the selected region has been scanned. Eachbitplane of the selected region above a minimum bit level is scanned.

[0173] SWEET also provides a method of encoding a digital image, themethod comprising the steps of:

[0174] decomposing the image using a discrete wavelet transform toprovide a number of subbands;

[0175] for each subband, selecting the subband as an initial region andperforming the following sub-steps:

[0176] (a) checking if a current bit level of the selected region issignificant;

[0177] (b) if the current bit level is significant, outputting a firsttoken in a coded representation and partitioning the selected regioninto a number of equally sized sub-regions, wherein each sub-region isprocessed as the selected region in turn;

[0178] (c) if the current bit level is insignificant, outputting asecond token in the coded representation and selecting the next lowerbit level of the selected region as the current bit level;

[0179] (d) repeating steps (a) to (c) until the current bit level isless than a specified minimum bit level, or the selected region has apredetermined size and coefficients of the selected region are coded inthe coded representation.

[0180] Preferably, the coefficients of the selected region are coded byrepresenting each coefficient by the bits between the correspondingcurrent bit level and the minimum bit level.

[0181] SWEET also provides a method of encoding a digital image, themethod comprising the steps of:

[0182] a) dividing the digital image into a plurality of blocks;

[0183] b) applying a subband transform to each of the blocks to providea plurality of AC subband regions, at one or more resolutions, and a DCsubband region for each block;

[0184] c) selection the DC subband region as a selected region andperforming the following sub-steps:

[0185] ca) checking if a current bitplane of the selected region issignificant;

[0186] cb) if the current bitplane is significant, outputting a firsttoken in a coded representation and partitioning the selected regioninto a number of sub-regions, wherein each sub-region is processed asthe selected region in turn;

[0187] cc) if the current bitplane is insignificant, outputting a secondtoken in the coded representation and selecting the next lower bitplaneof the selected region as the current bitplane;

[0188] cd) repeating sub-steps ca) to cc) until the current bitplane isless than a specified minimum bitplane, or the selected region has apredetermined size and coefficients of the selected region are coded inthe coded representation;

[0189] d) selecting substantially all uncoded AC subband regions as aremaining region of each block and scanning the significance of eachbitplane of the remaining region from a most significant bitplanetowards a least significant bitplane, and outputting the second tokenfor each insignificant bitplane until a significant bitplane isdetermined;

[0190] e) setting one or more AC subband regions, of a currentresolution level, as a selected region and performing the sub-steps ca)to cd);

[0191] f) repeating step e) until substantially all AC subbands of thecurrent resolution level have been encoded; and

[0192] g) repeating steps d) to f) until all AC subbands of each blockhave been encoded.

[0193] Other aspects of SWEET include a method for decoding the codedrepresentation of a digital image, an apparatus for representing adigital image to provide the coded representation, an apparatus fordecoding the coded representation of a digital image, a computersoftware system for representing a digital image to provide the codedrepresentation, and a computer software system for decoding the codedrepresentation, in which the coded representation is produced inaccordance with the first or second aspect of the invention.

[0194] A high-level block diagram is illustrated in FIG. 22 to providean overview of the embodiments of the SWEET methods. An input image 190is provided to the transform block 192, which is preferably a lineartransform, to produce corresponding transform coefficients 194. Adiscrete wavelet transform (DWT) is preferably employed.

[0195] The two-dimensional DWT of an image is a transform thatrepresents the image using a low frequency approximation to the imageand three high frequency detail components. Conventionally, thesecomponents are termed subbands. Each of the four sub-images formed bythe DWT is one quarter of the size of the original image. The lowfrequency image contains most of the information about the originalimage. This information, or energy compaction, is the feature of thediscrete wavelet transform image subbands that is exploited for imagecompression.

[0196] The single-level DWT can be applied recursively to the lowfrequency image, or subband, an arbitrary number of times. For example,a three-level DWT of the image is obtained by applying the transformonce and then applying the DWT to the low subband resulting from thetransformation. Thus, this results in 9 detailed subbands and one (very)low frequency subband. Even after three levels of DWTs, the resultinglow frequency subband still contains a significant amount of informationof the original image, yet is 64 times smaller (¼×¼×¼), therebyeffecting a factor of 64 in compression.

[0197] However, other linear transformations for decorrelating imagedata may be practiced without departing from the scope of the invention.For example, a discrete cosine transform (DCT) can be practiced. Thetransform coefficients 194, or more specifically the bit sequencesrepresenting their values, are then coded by the bit rearrangement block196 in an efficient fashion to provide the coded representation 198.

[0198] The decoding process is simply the reverse of this encodingprocess. The encoded coefficients are decoded into the transformcoefficients. The (transform domain) image is then inverse transformedto form the original image, or some approximation thereof.

[0199] SWEET is preferably be practiced using a conventionalgeneral-purpose computer, such as that shown in FIGS. 1, 2 and 3,wherein the processes of FIGS. 24 to 27 or FIGS. 29 to 32 areimplemented as software executing on the computer. In particular, thesteps of the coding and/or decoding methods are effected by instructionsin the software that are carried out by the computer system 900.

[0200] Before proceeding with a further description of the SWEETimplementation, a brief review of terminology used hereinafter isprovided. For a binary integer representation of a number, “bit n” or“bit number n” refers to the binary digit n places to the left of theleast significant bit. For example, assuming an 8-bit binaryrepresentation, the decimal number 9 is represented as 00001001. In thisnumber, bit 3 is equal to 1, while bits 2, 1, and 0 are equal to 0, 0,and 1, respectively.

[0201] For transform coding applications, the number of bits percoefficient required to represent the possible range of coefficients isdetermined by the linear transform and the resolution of each pixel (inbits per pixel) in the input image. This range of values for each pixelis typically large relative to the values of most of the transformcoefficients, and thus many coefficients have a large number of leadingzeros. For example, the number 9 has four leading zeros in a 8-bitrepresentation and has 12 leading zeros for a 16-bit representation. Theembodiment of the invention provides a method and apparatus forrepresenting (or coding) these leading zeros, for blocks ofcoefficients, in an efficient manner. The remaining bits and sign of thenumber are encoded directly without modification.

[0202] To simplify the description and not to obscure unnecessarily theinvention, the transform coefficients are assumed hereinafter to berepresented in an unsigned binary integer form, with a single sign bit.That is, the decimal numbers −9 and 9 are represented with the same bitsequence, namely 1001, with the former having a sign bit equal to 1 toindicate a negative value, and the latter having a sign bit equal to 0to indicate a positive value. The number of leading zeros is determinedby the range of the transform coefficients. In using an integerrepresentation, the coefficients are implicitly already quantised to thenearest integer value, although this is not necessary for thisembodiment of the invention. Further, for the purpose of compression,any information contained in fractional bits is normally ignored.

[0203] A region consists of a set of contiguous image coefficients. Theterm coefficient is used hereinafter interchangeably with pixel,however, as will be well understood by a person skilled in the art, theformer is typically used to refer to pixels in a transform domain (eg.,a DWT domain).

[0204] Encoding Process of SWEET

[0205]FIG. 24 is a flow diagram illustrating the image encoding methodaccording to the preferred embodiment. In step 302, processing commencesusing an input image. In step 304, the input image is transformed usinga linear transformation, preferably a discrete wavelet transform. Aninitial region is defined to be the whole image. For example, in thecase of a three-level DWT of the input image, the resulting coefficientsconsisting of the 10 subbands can be specified as the region.Alternatively each subband can be processed separately, setting eachinitial region to the whole subband in question.

[0206] In step 306, the most significant bit (msb) of the largestabsolute value of the transform coefficients is determined and aparameter, maxBitNumber, is set to this coefficient value. For example,if the largest transform coefficient has a binary value of 00001001(decimal 9), the parameter maxBitNumber is set to 3, since the msb isbit number 3. Alternatively, the parameter maxBitNumber may be set to beany value that is larger that the msb of the largest absolute value ofthe transform coefficients.

[0207] Further, in step 306, a coding parameter, minBitNumber is set tospecify the coded image quality. In particular, this coding parameterspecifies the precision of every coefficient in the transformed imageand can be varied as required. For example, a minBitNumber of 3 providesa coarser reproduction of the original image than does a value of 1.

[0208] Optionally, the technique involves step 308 which provides anoutput header in the coded representation of the input image. Thus, in apractical implementation, header information is output as part of thecoded representation. For example, the output header of the embodimentof the invention may contain information about the source image,including the image height and width, the number of levels of the DWT,the mean value of the DC subband, the maxBitNumber parameter, and theminBitNumber parameter.

[0209] Beginning in step 310, each subband of the transformed image iscoded separately in steps 312 and 314. Each subband is codedindependently, in order from low frequency to high frequency. For the DCsubband, the mean value is removed prior to coding and coded into theheader information in step 308. In step 312, each subband is coded bysetting an initial region as the whole subband. In step 314, the regionis encoded with the maxBitNumber and minBitNumber as parameters. Thisprovides a hierarchal code, since lower resolution versions of the imageare coded into the bit stream before higher resolutions. Processingterminates in step 316.

[0210]FIG. 25 is a detailed flow diagram of the procedure “Coderegion(currentBitNumber, minBitNumber)” called in step 314 of FIG. 24for coding each region, where maxBitNumber is provided as thecurrentBitNumber. In step 402, processing commences. The inputs to theregion coding process of FIG. 25 include the currentBitNumber andminBitNumber parameters. Preferably, the method is implemented as arecursive technique where the process is able to call itself with aselected region or sub-region. However, the process may implemented in anon-recursive manner without departing from the scope and spirit of theinvention.

[0211] In decision block 404, a check is made to determine if thecurrentBitNumber parameter is less than the minBitNumber parameter.Otherwise, if decision block 404 returns true (yes), nothing is done andprocessing returns to the calling procedure in step 406. This conditionindicates that every coefficient in the selected region has a msb numberless than minBitNumber. If decision block 404 returns false (no),processing continues at decision block 408.

[0212] In decision block 408, a check is made to determine if theselected region is a 1×1 pixel. If decision block 408 returns true(yes), processing continues at step 410. In step 410, the 1×1 pixel iscoded. Preferably, this involves directly outputting the remaining bitsabove the minBitNumber in the coded representation. In step 412,processing returns to the calling procedure. Otherwise, if decisionblock 408 returns false (no), the region consists of more than onecoefficient and processing continues at decision block 414.

[0213] In decision block 414, the selected region is checked todetermine if it is significant. That is, the significance of the regionis tested. The region is said to be insignificant if the msb number ofeach coefficient in the region is less than the value of thecurrentBitNumber parameter. To make the concept of region significanceprecise, a mathematical definition is given in Equation (1). At a givenbit number, say currentBitNumber =n, the region is said to beinsignificant if:

|c _(ij)|<2^(n) ,∀i,jεR,  (1)

[0214] where R denotes the region, and c_(ij) denotes coefficient (ij)in this region.

[0215] If decision block 414 returns false (no), processing continues atstep 416. In step 416, a value of 0 (or first token) is output in thecoded representation stream, and the currentBitNumber parameter isdecremented by 1. That is, the next, lower bitplane of the region isselected for processing. Processing then continues at decision block404, where the region is again processed with the parameterscurrentBitNumber−1 and minBitNumber. Otherwise, if decision block 414returns true (yes), that is, the region is significant, processingcontinues at step 418.

[0216] In step 418, a value of 1 (or second token) is output in thecoded representation stream. In step 420, the selected region ispartitioned into a predetermined number (preferably, 4) of subregionsusing a specified partitioning algorithm. The partitioning algorithmused is known to the decoder.

[0217] In this embodiment of the invention, square regions are used. Aregion is partitioned preferably into 4 equal-sized (square) subregions.As shown in FIG. 23, the selected region (R) 200 has a size of M×Mcoefficients and is partitioned into four equal-sized subregions 210,212, 214 and 216. Each of the subregions has a size of N×N, where N isequal to M/2. This is not always possible depending on the size andshape of the initial region. If this is not possible, the initial regioncan be partitioned into a number of square regions, each havingdimensions that are a power of 2, and encode these partitionsseparately. In any case, this initialisation has minimal effect on theoverall results if done in an intelligent fashion. In an alternateembodiment, a different partition may be used that is suitable for ablock-based coder.

[0218] In step 422, each subregion is then coded with the samecurrentBitNumber and minBitNumber parameters. This is preferably done bymeans of a recursive call to the procedure “Coderegion(currentBitNumber, minBitNumber)” of FIG. 25. This coding ofsubregions may be implemented in parallel or sequentially. In the lattercase, the processing may commence from a low frequency subband to higherfrequency subbands in turn.

[0219] In the coded representation, a transform coefficient is coded bysimply outputting the pixel bits from the currentBitNumber to theminBitNumber. Preferably, a convention is followed whereby the sign isoutput only if some of the coefficient bits were non-zero. For example,if currentBitNumber=3, minBitNumber=1, then −9 (00001001) is coded as “10 0” followed by a sign bit “1”.

[0220] Decoding Process of Preferred Embodiment

[0221]FIG. 26 is a flow diagram illustrating a method of decoding thecoded representation of an image obtained using the process of FIGS. 24and 25. In step 502, processing commences using the codedrepresentation. In step 504, the header information is read from thecoded representation to determine the size of the original image, andhence the initial region size. Also, information such as maxBitNumber(equal to the initial currentBitNumber in the coding process) andminBitNumber are input. Further information includes the mean value ofthe DC subband.

[0222] In step 506, decoding of each subband is commenced by setting theregion to the respective subbands in turn. In step 508, the selectedregion is decoded using the maxBitNumber and minBitNumber parameters. Instep 510, the inverse DWT is applied to the decoded selected region.Processing terminates in step 512.

[0223]FIG. 27 is a detailed flow diagram of step 508 of FIG. 26 fordecoding each region using procedure call “Decoderegion(currentBitNumber, minBitNumber)”, where maxBitNumber is providedas the currentBitNumber. In step 602, processing commences. The inputsto the region decoding process of FIG. 27 are the currentBitNumber andminBitNumber parameters. Again, the method is preferably implemented asa recursive technique. However, the process may be implemented in anon-recursive manner without departing from the scope and spirit of theinvention.

[0224] In decision block 604, a check is made to determine if thecurrentBitNumber is less than the minBitNumber. If decision block 604returns true (yes), processing continues at step 606, where processingreturns to the calling procedure. Otherwise, if decision block 604returns false (no), processing continues at decision block 608.

[0225] In decision block 608, a check is made to determine if theselected region has a size of 1×1 pixels. If decision block 608 returnstrue (yes), processing continues at step 610. In step 610, the 1×1region is decoded. Processing then returns to the calling procedure instep 612. If decision block 608 returns false (no), processing continuesat step 614. In step 614, a bit is input from the coded representation.

[0226] In decision block 616, a check is made to determine if the bit isequal to 1, that is, the input is checked to determine if the region issignificant. If decision block 616 returns false (no), processingcontinues at step 618. In step 618, the currentBitNumber is decremented,and processing continues at decision block 604. Otherwise, if decisionblock 616 returns true (yes), processing continues at step 620. In step620, the region is partitioned into the predetermined number(preferably, 4) of sub-regions. In step 622, each of the sub-regions isdecoded using the currentBitNumber and minBitNumber. In the preferredembodiment, this is carried out by means of a recursive call to theprocess illustrated in FIG. 27. In step 624, processing returns to thecalling procedure.

[0227] Thus, the bits output from the significance decisions in theencoder instruct the decoder on which path of the algorithm to take,thus mimicking the encoder. The pixels, and possible sign, are decodedby simply reading in the appropriate number of bits (currentBitNumber tominBitNumber and if some of these are non-zero the sign bit).

[0228] Two-Dimensional Example

[0229] The method effectively codes the leading zeros of most transformcoefficients, while coding the bits from the most significant bit to thepredetermined least significant bit, specified by the parameterminBitNumber, and the sign simply as is. Thus, the preferred embodimentof the present invention advantageously represents the leading zeros.This method is very efficient in certain situations, namely for codingdiscrete wavelet transform image coefficients, which typically exhibit alarge dynamic range. A few coefficients typically have very largevalues, while most have very small values.

[0230] An example of encoding a two-dimensional region comprising 4×4coefficients is described with reference to FIGS. 28A to 28D. Theprocessing of the 4×4 region 700 of FIG. 28A is commenced with themaxBitNumber set to 7 since this is the-largest bit number (bitplane) ofall of the coefficients: $\quad\begin{bmatrix}200 & 13 & {- 11} & {- 8} \\{- 13} & 3 & {- 4} & {- 3} \\8 & 1 & {- 2} & {- 2} \\2 & {- 1} & {- 3} & {- 3}\end{bmatrix}$

[0231] The minBitNumber is set to 3, for illustrative purposes. A headeris preferably output in the coded representation containing themaxBitNumber and minBitNumber. The process of coding the region 700 thenfollows.

[0232] At currentBitNumber=7, a one (1) is output since the region 700is significant with respect to bit number 7 (see decision block 404,408, and 414 and step 418 of FIG. 25). The region 700 is thenpartitioned into four sub-regions (see step 420 of FIG. 25): the topleft region 710, the top right region 712, the bottom left region 714and the bottom right region 716 of FIG. 28A. Each of the subregionsconsist of 2×2 coefficients.

[0233] The sub-regions 710, 712, 714 and 716 of FIG. 28A are in turncoded in the predefined processing sequence shown of FIG. 28B, where aregion 750 consists of four sub-regions 750A to 750D. The three arrowsillustrated in the diagram indicate the order or sequence of processing,that is, top left sub-region 750A, top right sub-region 750B, bottomleft sub-region 750C and bottom right sub-region 750D, respectively.

[0234] The sub-region 710 of FIG. 28A is coded first (see step 422 ofFIG. 4). For the currentBitNumber equal to 7, a one (1) is output in thecoded representation. The sub-region 710 is then partitioned into four1×1 pixels having decimal values 200, 13, −13 and 3. Each of thesecoefficients is coded by outputting the bits of each coefficient fromthe currentBitNumber=7 to the minBitNumber=3 (see decision block 408 andstep 410 of FIG. 28). A sign bit is then output if required. Thus, thedecimal value is 200 is coded as 11001 followed by the sign bit 0. Thecoefficient value 13 is coded as 00001 with a sign bit 0. Thecoefficient value −13 is coded as 00001 with a sign bit 1. Finally, thecoefficient value 3 is coded as 00000 (without a sign bit). The codedrepresentation of each coefficient includes the two “1” bits precedingthe bits of coefficients “200” between the currentBitNumber andminBitNumber. This completes the coding of the top left sub-region 710.The coded output at this state is:$1\underset{\underset{200}{}}{1\quad 1001}\quad \overset{\overset{{sign}\quad {bit}}{}}{0}\quad \underset{\underset{13}{}}{00001}0\underset{\underset{- 13}{}}{00001}1{\underset{\underset{3}{}}{00000}.}$

[0235] The header information is not shown in the foregoing expression.

[0236] The top right sub-region 712 is then coded (per FIG. 28B). A zero(0) is output for each of currentBitNumber equal to 7, 6, 5, and 4,since the region 712 is insignificant with respect to these bit numbers.A one (1) is output at currentBitNumber=3, since this bitplane issignificant with respect to bit number 3. The sub-region 712 ispartitioned into the four 1×1 pixels having values −11, −8, −4 and −3.These decimal values are coded as bit value 1 with sign bit 1, bit value1 with sign bit 1 and bit values 0 and 0 without sign bits,respectively. Thus, at this stage, the coded representation is asfollows:$11100100000100000110000000001\quad \underset{\underset{- 11}{}}{11}\quad \underset{\underset{- 8}{}}{11}\quad \underset{\underset{{- 4}\quad - 3}{\bullet \bullet \bullet \bullet}}{0\quad 0}$

[0237] The bottom left sub-region 714 is then encoded. A zero (0) isoutput for each of currentBitNumber equal to 7, 6, 5, and 4, since theregion 714 is insignificant with respect to these bit numbers. A one (1)is output at currentBitNumber equal to 3, since this bitplane issignificant with respect to bit number 3. The sub-region 714 is thenpartitioned into four 1×1 pixels having values 8, 1, 2 and −1. These arecoded respectively as binary value 1 with sign bit 0, and binary values0,0 and 0 without sign bits.

[0238] Finally, the bottom right sub-region 716 having values −2, −2,−3, and −3 is coded. A zero (0) is output for each ofcurrentBitNumber=7, 6, 5, 4 and 3 since the sub-region 716 isinsignificant with respect to these bit numbers. No sign bits areoutput. Thus, the coded representation is as follows:

111100100000100000110000000001111100000011000000000.

[0239] The decoder simply mimics the encoding process to reconstruct theregion from the coded representation as depicted in FIG. 28C.

[0240] The decoding process can be made “smarter” in a number of ways.One such a “smarter” way is depicted in FIG. 28D. In this case, themagnitude of the non-zero coefficients is each increased by half of 2 tothe power of minBitNumber. This is depicted in FIG. 28D. In this manner,the “smart” decoding processing generally reduces the mean square errorbetween the decoded and the original coefficients. Still further, theencoder can alternatively perform this (type of) operation, therebyleaving the decoder to use the simplest depicted in FIG. 28C.

[0241] Encoding Process of Alternate SWEET

[0242] A coding process according to an alternate SWEET method ishereinafter described with reference to FIGS. 29 to 32. The processesillustrated in the flow diagrams of FIGS. 29 to 32 may be implementedusing software executing on a general-purpose computer 900 correspondingto that described.

[0243] A discrete wavelet transform of an entire digital image can beperformed on a block-by-block basis. The result of the transformationupon each block is a set of coefficients, which are essentiallyequivalent to a set of spatially corresponding coefficients of adiscrete wavelet transform of the entire image. For example, from apredetermined set of coefficients of a DWT for an entire image, aportion or block of the digital image can be reproduced to a specifieddetail. Selecting the predetermined set of coefficients from thefrequency domain amounts substantially to representing the correspondingportion of a digital image (the block) from the spatial domain. A blockbased DWT of a digital image can be performed by decomposing an imageinto a plurality of blocks and applying the transform to each blockindependently, thereby substantially evaluating those DWT coefficientsrelevant to the current spatial location. The advantage of adopting ablock-based transform approach is that a block can be subsequentlyencoded with minimal interaction (substantially independent) fromanother block of the image. Block-based techniques are inherently memorylocalised and therefore are generally efficient when implemented usingcomputer systems.

[0244]FIG. 29 is a flow diagram illustrating the block-based encodingprocess according to the alternate embodiment of the invention.Processing commences at step 902. In step 904, a header is output. Thisinformation preferably includes the image height and width, the blocksize, the number of levels of the DWT, and two coding parametersmaxBitNumber and minBitNumber. Optionally, more or less headerinformation may be used depending upon the application.

[0245] The coding parameter maxBitNumber can be selected in a variety ofways. If the block DWT is performed on all image blocks prior to codingof any of them, the maxBitNumber can be chosen to be the MSB number ofthe largest coefficient across all DWT blocks. For example, if thelargest coefficient is 10000001 (decimal value 129), the maxBitNumber isset to 7 since the MSB is bit number 7. Alternatively, a deterministicbound can be used which is determined by the transform and theresolution of the input image. For example, with an 8-bit input image(level shifted to 7-bits plus sign) and the Haar transform, the largestMSB is bounded by J+7 where J is the number of levels of the DWT. If theblocks are small, the selection of this parameter can have a significanteffect on compression. In some instances, more sophisticated ways ofselecting maxBitNumber may be employed. However, this depends upon thespecific application.

[0246] The parameter minBitNumber determines the compression ratioversus quality trade off and can be varied. For example, for nearlyorthogonal transforms, a value of 3 provides adequate image quality for8-bit, grey-scale or 24-bit, RGB images.

[0247] In step 906, the image is decomposed into blocks (or an imageblock is formed). The image is decomposed preferably into overlappingblocks. However, non-overlapping blocks may be employed. The block ofcoefficients can be as large as the whole original image, or as small asa block of 8×8 coefficients (for a three-level transform). For lowmemory applications, a block that is as small as possible may beemployed. Generally, a block size of 16 coefficients is sufficient forhigher levels of compression with a three or four level DWT. A blocksize of 8×8 coefficients with a three-level DWT can maintain good codingefficiency by employing differential pulse code modulation (DPCM) on theDC coefficient of each block.

[0248] In step 908, each block is level shifted and the transform isperformed. Preferably, a DWT is employed. The image values are levelshifted (for example, by 128 for an 8-bit image) to reduce or eliminateany undue mean bias, and each spatial block of the image is transformed.For a DWT, usually some knowledge of the block surrounding the currentblock is needed (and similarly for the inverse DWT), although this isnot strictly required.

[0249] In step 910, the block is coded using the maxBitNumber andminBitNumber parameters. Processing terminates in step 912.

[0250] Step 910 for coding a block is illustrated in detail in the flowdiagram of FIG. 30. Inputs to the block coding process of FIG. 30include the currentBitNumber and the minBitNumber parameters. Withreference to step 910 of FIG. 29, the maxBitNumber is input as thecurrentBitNumber parameter. Processing commences in step 1002. Indecision block 1004, a check is made to determine if thecurrentBitNumber is less than the minBitNumber. If decision block 1004returns true (yes), processing continues at step 1006. In step 1006,execution returns to the calling process, thereby indicating that everycoefficient in the block has an MSB number less than the minBitNumber.Otherwise, if decision block 1004 returns false (no), processingcontinues at decision block 1008.

[0251] In decision block 1008, a check is made to determine if a currentblock is significant. If decision block 1008 returns false (no),processing continues at step 1010. In step 1010, a zero (0) is output inthe coded representation and the currentBitNumber is decremented, thatis, the next lower bit plane is selected. Processing then continues atdecision block 1004. Otherwise, if decision block 1008 returns true(yes) processing continues at step 1012.

[0252] Decision blocks 1004 and 1008 along with step 1010 enable theprocess to find the MSB number of the largest coefficient in the block.A block is insignificant with respect to the currentBitNumber if the MSBnumber of every coefficient in the block is less than thecurrentBitNumber. This is repeated until the bitplane of the block issignificant or the currentBitNumber is less than the minBitNumber.

[0253] In step 1012, a one (1) is output in the coded representation toindicate the bitplane is significant. In step 1014, the DC subband iscoded. In step 1016, the block detail is coded using the parameters J,currentBitNumber and minBitNumber. In step 1018, execution returns tothe calling procedure. Thus, given that the block is significant, steps1012, 1014 and 1016 are carried out to use the (generalised) quadtreesegmentation to find all coefficients with an MSB number greater thanthe minBitNumber. If the block is significant, it is partitioned intotwo “sub-blocks”: the DC subband coefficients and the block consistingof the remaining coefficients, referred to as the “block detail” forlevel J since it represents the high frequency information about theblock of level J at all lower levels.

[0254] Step 1014 of FIG. 30 for coding the DC subband is illustrated indetail by the flow diagram of FIG. 32. That is, FIG. 32 shows theprocess of coding a subband or sub-block using currentBitNumber andminBitNumber parameters. In step 1202, processing commences. In decisionblock 1204, a check is made to determine if the currentBitNumber is lessthan the minBitNumber. If decision block 1204 returns true (yes),processing continues at step 1206. In step 1206, execution returns tothe calling procedure. Otherwise, if decision block 1204 returns false(no), processing continues at decision block 1208.

[0255] In decision block 1208 a check is made to determine if the(subband) block size is 1×1 pixels. If decision block 1208 returns true(yes), processing continues at step 1210. In step 1210, the 1×1 pixel iscoded. This involves outputting the bits between the currentBitNumberand the minBitNumber, inclusive, followed by a sign bit if necessary.Processing then returns to the calling procedure in step 1212.Otherwise, if decision block 1208 returns false (no), processingcontinues at decision block 1214.

[0256] In decision block 1214, a check is made to determine if the(subband) block is significant. If decision block 1214 returns false(no), processing continues at step 1216. In step 1216, a zero (0) isoutput in the coded representation and the currentBitNumber isdecremented. Processing then continues at decision block 1204.Otherwise, if decision block 1214 returns true (yes), processingcontinues at step 1218.

[0257] In step 1218, a one (1) is output in the coded representation toindicate that the (subband) block is significant. In step 1220, the(subband) block is partitioned into four sub-blocks. In step 1222, eachsub-block is coded using the parameters currentBitNumber andminBitNumber, by means of a recursive call to the process of FIG. 32. Instep 1224, execution returns the calling procedure.

[0258] Thus, in the process of FIG. 32, a subband or sub-block thereofis coded. The largest MSB number is isolated as before. If the sub-blockconsists of only one pixel, it is coded as a single coefficient.Otherwise, the currentBitNumber is decremented and a zero (0) is outputin the coded representation until the currentBitNumber is less than theminBitNumber, or the subband (sub-block) is significant. If the subband(sub-block) is significant, it is partitioned into four (as close toequal as possible) sub-block, and these are coded in turn. A singlecoefficient, for example the DC coefficient, is encoded by outputtingthe coefficient bits from the currentBitNumber to the minBitNumber.Again, the sign is preferably only output if some of the coefficientbits are non-zero.

[0259] Step 1016 of FIG. 30 for coding block detail is illustrated bythe flow diagram of FIG. 31. In step 1102, processing commences. Indecision block 1104, a check is made to determine if thecurrentBitNumber is less than the minBitNumber. If decision block 1104returns true (yes), execution returns to the calling procedure in step1106. Otherwise, if decision block 1104 returns false (no), processingcontinues at decision block 1108.

[0260] In decision block 1108, a check is made to determine if the block(detail) is significant. If decision block 1108 returns false (no),processing continues at step 1110. In step 1110, a zero (0) is output inthe coded representation and the currentBitNumber is decremented.Processing then continues at decision block 1104. Otherwise, if decisionblock 1108 returns true (yes), processing continues at step 1112.

[0261] In step 1112, a one (1) is output in the coded representation toindicate that the block (detail) is significant. In step 1114, each ofthe high-low (HL), low-high (LH), and high-high (HH) frequency subbandsis coded. The HL, LH, and HH frequency subbands of each resolution arecommonly referred to as AC subbands. Each of these subbands is coded inaccordance with the process of FIG. 12. In step 1116, the block detailis coded using the parameters J−1, currentBitNumber and minBitNumber (ifthe block detail exists) by means of a recursive call to the processillustrated in FIG. 11. Execution returns to the calling procedure instep 1118.

[0262] Thus, the block detail for level J is processed to first isolatethe MSB number of the largest coefficient. This is done by decrementingthe currentBitNumber and outputting zeros until the block issignificant. The block is then partitioned into the three high frequencysubbands at level J and the block detail for level J−1 (if J−1 isgreater than 0). This partitioning approach is motivated by the socalled 1/f type spectral models.

[0263] The decoding process for the alternate SWEET method can beimplemented by mimicking the coding process described with reference toFIGS. 29 to 32.

[0264] SWEET therefore provides methods and apparatuses for representingdigital image data in an efficient and flexible manner, in which therepresentation is suitable for storing and/or transmitting images. Theencoding techniques can be used generally to represent an array oftransform coefficients, and to provide an efficient representation byrepresenting an image in the discrete wavelet transform domain. Inparticular, the embodiments provide methods and apparatuses forrepresenting (or coding) leading zeros of blocks of transformcoefficients obtained from an input image. The techniques are efficientin terms of offering a good reproduction of the original image for agiven size code and offering fast decoding. Further, the techniques areflexible in that coefficients obtained from a linear transformation areencoded independently without the use of entropy coding. Theadvantageous aspects of the embodiment include the depth first nature ofthe coding. Further, in the case of coding subbands, the advantageousaspects of the invention include hierarchical coding of each subbandseparately.

SUMMARY

[0265] It will be apparent from the foregoing that a number ofarrangements are disclosed that provide for the manipulation of images,and in particular thumbnail images using fast zooming, aspect ratiozooming as well as compression and scaling. Such methods find particularapplication in image-based file management systems and browsing-typeapplications used in computer systems and the like. However, suchapplications should be construed only as exemplary of the variousdescribed embodiments.

[0266] The foregoing only describes a small number of embodiments of thepresent invention, however, modifications and/or changes can be madethereto by a person skilled in the art without departing from the scopeand spirit of the invention.

We claim:
 1. A method of displaying a representation of a plurality ofimages concurrently on a display device, each said image being encodedin a hierarchical compressed format, said hierarchical compressed formatadapted to provide, without substantially compromising compressionefficiency, a decoding of each said encoded image into one or more of aplurality of predetermined sizes, each size definable by a number ofpixels, said method comprising the steps of: decoding each encoded imageof said plurality of images into at least one of said plurality ofpredetermined sizes, wherein a decode time for each encoded image ateach one of said predetermined sizes is substantially proportional tothe number of pixels of each decoded image; and displaying each decodedimage of said plurality of images concurrently on said display device.2. The method according to claim 1, wherein each said decoded image is athumbnail representation of each said corresponding encoded image. 3.The method according to claim 1, wherein each said encoded image isencoded using a compression technique employing a linear transform. 4.The method according to claim 3, wherein said compression techniquesubstantially encodes groups of coefficients representing a resolutionof said linear transform substantially independently of coefficientsrepresenting another resolution.
 5. The method according to claim 3,wherein said linear transform is a discrete wavelet transform.
 6. Themethod according to claim 1, wherein said plurality of predeterminedsizes comprises a set of sizes related by powers of
 2. 7. The methodaccording to claim 1, wherein said plurality of predetermined sizes isthe same for each decoded image in said plurality of images.
 8. Themethod according to claim 1, wherein said decoding step and saiddisplaying step are interleaved for each image of said plurality.
 9. Amethod of displaying a plurality of images concurrently on a displaydevice, said method comprising the steps of: encoding each image in ahierarchical compressed format capable of providing a plurality ofpredetermined sizes of each said image without substantiallycompromising compression efficiency, wherein each image is encoded in amanner so that a decode time for each image at each of saidpredetermined sizes is substantially proportional to the number ofpixels in each decoded image, decoding each image of said plurality ofimages, and displaying each decoded image of said plurality of imagesconcurrently on said display device.
 10. The method according to claim9, wherein each said decoded image is a thumbnail representation of eachcorresponding encoded image.
 11. The method according to claim 9,wherein each said encoded image is encoded using a compression techniqueemploying a linear transform.
 12. The method according to claim 11,wherein said compression technique substantially encodes groups ofcoefficients representing a resolution of said linear transformsubstantially independently of coefficients representing anotherresolution.
 13. The method according to claim 11, wherein said lineartransform is a discrete wavelet transform.
 14. The method according toclaim 9, wherein said plurality of predetermined sizes comprises a setof sizes related by powers of
 2. 15. The method according to claim 9,wherein said plurality of predetermined sizes is the same for eachdecoded image in said plurality of images.
 16. The method according toclaim 9, wherein said decoding step and said displaying step areinterleaved for each image of said plurality.
 17. The method accordingto claim 2 or 10, wherein a predetermined size of each decoded image isequal to or greater than a desired size of said thumbnail.
 18. Themethod according to claim 17, further comprising the step ofdownsampling each decoded image to provide the desired-size thumbnail ifthe size of said decoded image is greater than said desired size of saidthumbnail.
 19. The method according to claim 17, further comprising thestep of upsampling the decoded image to provide the desired-sizethumbnail if the size of said decoded image is less than the desiredsize of said thumbnail.
 20. The method according to claim 2 or 10,further comprising the step of decoding said image at two differentadjacent sizes of a plurality of sizes to provide said image having adesired size.
 21. The method according to claim 20, further comprisingthe step of interpolating between the two decoded images havingdifferent adjacent sizes to provide the desired-size thumbnail if thesize of one of said two decoded images is greater than the desiredthumbnail size and the other of said two decoded images is less than thedesired thumbnail size.
 22. The method according to claim 5 or 13,wherein each image is encoded by the steps of: transforming said imageto derive a plurality of coefficients, each coefficient represented by apredefined bit sequence; selecting a portion of said plurality ofcoefficients as a region; (a) scanning the significance of each bitplaneof said selected region from a most significant bitplane towards a leastsignificant bitplane and providing a first token in said codedrepresentation for each insignificant bitplane until a significantbitplane is determined, wherein a second token is provided in said codedrepresentation for said significant bitplane; (b) partitioning saidselected region into two or more subregions having a predetermined form,and setting each of said subregions as said selected region; (c)repeating steps (a) and (b) commencing from said significant bitplaneuntil said selected region has a predetermined size, wherein saidcoefficients of said selected region are coded and provided in saidcoded representation.
 23. The method according to claim 22, wherein saidtransforming step comprises applying a discrete wavelet transform tosaid image.
 24. The method according to claim 22, wherein said regioncomprises said entire plurality of coefficients.
 25. The methodaccording to claim 23, wherein said portion comprises a subband of saidplurality of coefficients.
 26. The method according to claim 22, whereinsaid first and second tokens comprise bit values of 0 and 1,respectively.
 27. The method according to claim 22, wherein saidsubregions are equally sized.
 28. The method according to claim 27,wherein said subregions are square.
 29. The method according to claim22, wherein said predetermined size of said subregion is a 1×1coefficient.
 30. The method according to claim 29, wherein said 1×1coefficient is encoded by outputting bits of said corresponding bitsequence beginning with the respective significant bitplane.
 31. Themethod according to claim 30, wherein only bits of said correspondingbit sequence above a predetermined minimum bit level are output in saidcoded representation.
 32. The method according to claim 22, wherein, instep (c), steps (a) and (b) are repeated otherwise until each bitplaneof said selected region has been scanned.
 33. The method according toclaim 32, wherein each bitplane of said selected region above a minimumbit level has been scanned.
 34. The method according to claim 5 or 13,wherein each image is encoded by the steps of: decomposing said imageusing a subband transform to provide a number of subbands; for eachsubband, selecting said subband as an initial region and performing thefollowing sub-steps: (a) checking if a current bit level of the selectedregion is significant; (b) if said current bit level is significant,outputting a first token in a coded representation and partitioning saidselected region into a number of equally sized sub-regions, wherein eachsub-region is processed as said selected region in turn; (c) if saidcurrent bit level is insignificant, outputting a second token in saidcoded representation and selecting the next lower bit level of saidselected region as said current bit level; (d) repeating steps (a) to(c) until said current bit level is less than a specified minimum bitlevel, or said selected region has a predetermined size and coefficientsof said selected region are coded in said coded representation.
 35. Themethod according to claim 34, wherein said coefficients of said selectedregion are coded by representing each coefficient by the bits betweenthe corresponding current bit level and the minimum bit level.
 36. Themethod according to claim 34, wherein said subband transform comprises adiscrete wavelet transform.
 37. The method according to claim 5 or 13,wherein each image is encoded by the steps of: a) dividing each saidimage into a plurality of blocks; b) applying a subband transform toeach of said blocks to provide a plurality of AC subband regions, at oneor more resolutions, and a DC subband region for each block; c)selection said DC subband region as a selected region and performing thefollowing sub-steps: ca) checking if a current bitplane of the selectedregion is significant; cb) if said current bitplane is significant,outputting a first token in a coded representation and partitioning saidselected region into a number of sub-regions, wherein each sub-region isprocessed as the selected region in turn; cc) if said current bitplaneis insignificant, outputting a second token in the coded representationand selecting the next lower bitplane of the selected region as thecurrent bitplane; cd) repeating sub-steps ca) to cc) until said currentbitplane is less than a specified minimum bitplane, or said selectedregion has a predetermined size and coefficients of said selected regionare coded in said coded representation; d) selecting substantially alluncoded AC subband regions as a remaining region of each block andscanning the significance of each bitplane of the remaining region froma most significant bitplane towards a least significant bitplane, andoutputting the second token for each insignificant bitplane until asignificant bitplane is determined; e) setting one or more AC subbandregions, of a current resolution level, as a selected region andperforming the sub-steps ca) to cd); f) repeating step e) untilsubstantially all AC subbands of the current resolution level have beenencoded; and g) repeating steps d) to f) until all AC subbands of eachblock have been encoded.
 38. The method according to claim 1 or 9,further comprising the step of caching said encoded image and/or saiddecoded image.
 39. The method according to claim 38, further comprisingthe step of caching intermediate size versions of said decoded image.40. Apparatus for displaying a representation of a plurality of imagesconcurrently on a display device, said apparatus comprising: means forstoring each said image, having being encoded in a hierarchicalcompressed format, said hierarchical compressed format adapted toprovide, without substantially compromising compression efficiency, adecoding of each said encoded image into one or more of a plurality ofpredetermined sizes, each size definable by a number of pixels; andmeans for decoding each encoded image of said plurality of images intoat least one of said plurality of predetermined sizes, wherein a decodetime for each encoded image at each one of said predetermined sizes issubstantially proportional to the number of pixels of each decodedimage; and means for displaying each decoded image of said plurality ofimages concurrently on said display device.
 41. The apparatus accordingto claim 40, wherein each said decoded image is a thumbnailrepresentation of each corresponding encoded image.
 42. The apparatusaccording to claim 40, wherein each said encoded image is encoded usinga compression technique employing a linear transform.
 43. The apparatusaccording to claim 42, wherein said compression technique substantiallyencodes groups of coefficients representing a resolution of said lineartransform substantially independently of coefficients representinganother resolution.
 44. The apparatus according to claim 42, whereinsaid linear transform is a discrete wavelet transform.
 45. The apparatusaccording to claim 40, wherein said plurality of predetermined sizescomprises a set of sizes related by powers of
 2. 46. The apparatusaccording to claim 40, wherein said plurality of predetermined sizes isthe same for each decoded image in said plurality of images.
 47. Theapparatus according to claim 40, wherein said decoding step and saiddisplaying step are interleaved for each image of said plurality.
 48. Animage-based computerized file browsing system comprising: a storagedevice for retaining a plurality of images at least group of which arestored in an encoded, hierarchical compressed format, said hierarchicalcompressed format adapted to provide, without substantially compromisingcompression efficiency, a decoding of each said image into one or moreof a plurality of predetermined sizes, each size comprising a number ofpixels, a processor for running an operating system incorporating a filemanagement arrangement wherein true images retained in said storagedevice are depicted as thumbnail representations when viewed via saidfile management system, characterised by an image manipulationarrangement whereby said images are decoded into at least one of saidplurality of sizes, wherein a decode time for each image at each of saidpredetermined sizes is substantially proportional to the number ofpixels of each decoded image.
 49. The system according to claim 48,wherein each said decoded image is a thumbnail representation of eachsaid corresponding encoded image.
 50. The system according to claim 48,wherein each said encoded image is encoded using a compression techniqueemploying a linear transform.
 51. The system according to claim 50,wherein said compression technique substantially encodes groups ofcoefficients representing a resolution of said linear transformsubstantially independently of coefficients representing anotherresolution.
 52. The system according to claim 50, wherein said lineartransform is a discrete wavelet transform.
 53. The system according toclaim 48, wherein said plurality of predetermined sizes comprises a setof sizes related by powers of
 2. 54. The system according to claim 48,wherein said plurality of predetermined sizes is the same for eachdecoded image in said plurality of images.
 55. The system according toclaim 48, wherein said decoding step and said displaying step areinterleaved for each image of said plurality.
 56. A computer programproduct comprising a computer readable medium having a computer programrecorded thereon for displaying a representation of a plurality ofimages concurrently on a display device, said computer program productcomprising: means for storing said plurality of images having beenencoded in a hierarchical compressed format, said hierarchicalcompressed format adapted to provide, without substantially compromisingcompression efficiency, a decoding of each said image into one or moreof a plurality of predetermined sizes, each size comprising a number ofpixels; means for decoding each of said images into at least one of saidplurality of sizes, wherein a decode time for each image at each of saidpredetermined sizes is substantially proportional to the number ofpixels of each decoded image; and means for displaying each decodedimage of said plurality of images concurrently on the display device.57. The computer program product according to claim 56, wherein eachsaid decoded image is a thumbnail representation of each saidcorresponding encoded image.
 58. The computer program product accordingto claim 56, wherein each said encoded image is encoded using acompression technique employing a linear transform.
 59. The computerprogram product according to claim 58, wherein said compressiontechnique substantially encodes groups of coefficients representing aresolution of said linear transform substantially independently ofcoefficients representing another resolution.
 60. The computer programproduct according to claim 58, wherein said linear transform is adiscrete wavelet transform.
 61. The computer program product accordingto claim 56, wherein said plurality of predetermined sizes comprises aset of sizes related by powers of
 2. 62. The computer program productaccording to claim 56, wherein said plurality of predetermined sizes isthe same for each decoded image in said plurality of images.
 63. Thecomputer program product according to claim 56, wherein said decodingstep and said displaying step are interleaved for each image of saidplurality.